-- Logs begin at Thu 2022-06-16 15:40:00 UTC, end at Thu 2022-06-16 17:05:20 UTC. -- Jun 16 16:26:01.047556 dv-cinder-837355-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 16:26:01.057150 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 16:26:01 2022] *** Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: nodename: dv-cinder-837355-2-iscsi Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: machine: x86_64 Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: clock source: unix Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: pcre jit disabled Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: detected number of CPU cores: 4 Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: current working directory: / Jun 16 16:26:01.059224 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: detected binary path: /usr/bin/uwsgi-core Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: your processes number limit is 47680 Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: your memory page size is 4096 bytes Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: detected max file descriptor number: 2048 Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: lock engine: pthread robust mutexes Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: thunder lock: enabled Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 16:26:01.059740 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 16:26:01.105555 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: Python main interpreter initialized at 0x555973c4f9c0 Jun 16 16:26:01.105555 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: python threads support enabled Jun 16 16:26:01.105555 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: your server socket listen backlog is limited to 100 connections Jun 16 16:26:01.105555 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: your mercy for graceful operations on workers is 90 seconds Jun 16 16:26:01.106063 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 16:26:01.106063 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: *** Operational MODE: preforking *** Jun 16 16:26:01.106288 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: *** uWSGI is running in multiple interpreter mode *** Jun 16 16:26:01.106288 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: spawned uWSGI master process (pid: 80329) Jun 16 16:26:01.106523 dv-cinder-837355-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 16:26:01.109033 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 1 (pid: 80339, cores: 1) Jun 16 16:26:01.109033 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 2 (pid: 80340, cores: 1) Jun 16 16:26:01.109033 dv-cinder-837355-2-iscsi devstack@keystone.service[80329]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 16:26:06.631790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 16:26:06.631790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: from cryptography.utils import int_from_bytes Jun 16 16:26:06.632612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 16:26:06.632612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: from cryptography.utils import int_from_bytes Jun 16 16:26:06.663778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 16:26:06.663778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Jun 16 16:26:06.664507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 16:26:06.664507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Jun 16 16:26:06.682664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 16:26:06.682836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 16:26:06.683038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] command line args: [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 16:26:06.683350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 16:26:06.683488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ================================================================================ {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 16:26:06.683771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] admin_token = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.684020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] config_dir = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.684184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.684458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] config_source = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.684590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] debug = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.684733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.685926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.685926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.685926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.685926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_config_append = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_dir = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_file = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.686899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.687177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.687424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.687675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.687836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.688020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.688271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.688521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.688738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.688981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.689198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.689442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.715703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 16:26:06.715921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 16:26:06.716106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] command line args: [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 16:26:06.716343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 16:26:06.716551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ================================================================================ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 16:26:06.716812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.717064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_dir = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.717226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.717413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_source = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] debug = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.717827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.718072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.718239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.718500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.718751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.718983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.719200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_config_append = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.719387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.719583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_dir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.719761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.719991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.720257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.720507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.720760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.720937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.721116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.721362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.721613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.721837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.722083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.722317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.722548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.722798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.777013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.777275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.777619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.778183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.778362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] publish_errors = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.778639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.778864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.779079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.779357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.779609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.779830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] standard_threads = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.780078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.780346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.780512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.780704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] use_journal = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.780882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] use_json = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.781057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] use_stderr = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.781236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] use_syslog = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.781470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.781725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.781973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.782243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.782490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.782745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.782989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.783274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.783519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.783773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.784026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.784290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.784529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.784780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.785020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.785271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.785526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.785768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.786028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.786284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.786537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.786776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.787034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.787330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.787590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.787847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.788082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.788317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.788552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.788792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.789052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.789376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.789615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.789853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.790105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.790601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.790855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.791135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.791360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.791603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.791839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.792074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.792308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.792542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.792785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.793040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.793277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.793518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.793753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.793994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.794240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.794493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.794726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.794967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.795236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.795477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.795718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.795958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.796192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.796447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.796687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.796927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.797163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.797429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.797669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.797954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.798144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.798384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.798662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.798906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.799192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.799449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.799694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.799927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.800166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.800415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.800654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.800893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.801139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.801378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.801616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.801853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.802093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.802329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.802599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.802844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.803108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.803352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.803597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.803836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.804080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.804319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.804553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.804785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.805046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.805283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.805516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.805858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.805997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.806248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.806487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.806751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.807011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.807280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.807514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.external = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.807746 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.808043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.808279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.808513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.password = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.808745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] auth.token = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.808988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.809226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.809473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.809706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.809956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.811859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.812096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.812350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.812606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.812845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.812976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.813100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.813335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.813447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.813628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.813894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.814032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.814139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] publish_errors = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.814249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.814382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.814494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.814608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.814748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.814864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.814994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.815115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.815220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.815323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.815472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.815583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] standard_threads = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.815716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.815820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.815903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.816033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.816143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.816394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_journal = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.816662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_json = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_stderr = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.816883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_syslog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.816975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 16:26:06.817225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.817984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.818946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.819946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.820955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.821985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.822984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.823956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.824939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.825997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.826968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.827919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.828926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.829928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.830933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.831949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.833996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.834906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.835925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.836958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.837929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.838944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.839925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.840979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] role.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] role.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.841995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.842967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.external = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.843944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.password = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.token = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.844940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.845891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.846978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.847915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.848973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.849979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.850941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.851851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.852933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.852933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.853895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.connection = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.854964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855394 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.855980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.856893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.857998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80340) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 16:26:06.858892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.858892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.859328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.859328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.859538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.859743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.859990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.860237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.860488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.860723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.860975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.861226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.861466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.861716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.861967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.862202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.862445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.862678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.862919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.863187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.863440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.863681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.863921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.864176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.864428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.864663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.864895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.865147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.865399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.865639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.865884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.866125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.866361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.866604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.866869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.867133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.867435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.868316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.868584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.868836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.869083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.869331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.869578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.869824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.870383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.870667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.870922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.871202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.871455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.871710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.871917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.872201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.872446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.872686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.872927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.873168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.873406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.873659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.873993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.874270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.874510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.874763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.875004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.875272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.875501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.875756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.876004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.876256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.876485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.876729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.876985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.877238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.877478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.877734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.877991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.878231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.878480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.878720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.878963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.879223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.879477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.879713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.879968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.880220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.880473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.880724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.880977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.881229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.881481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.881731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.881968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.882219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.882470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.882721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.883128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.884165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.884411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.884672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.884935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.885195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.885458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.885704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.885968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.886230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.886478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.886724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.886963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.887227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.887487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.887751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.888006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.888250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.888329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.888492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.888581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.888729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.888808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.888971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.889214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.889469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.889713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.889975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.890228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.890471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.890720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.890969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.891247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.891491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.891733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.891974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.892225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.892478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.892712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.892958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.893271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.893538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.893785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.894036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.894279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.894525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.894765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.895007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.895270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.895512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.895762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.896002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.896251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.896496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.896748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.896988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.897240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.897476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.897556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.897721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.897962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 16:26:06.898135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 16:26:06.924505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.924783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.924986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.925782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.926019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.934232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.953829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.954057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.982606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:06.994686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.032229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.032486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.036258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.036487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.100562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.100734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.100926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.101928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.102673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.125879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.126881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.138038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.138972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.150795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.150998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.151287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.151480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.151714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.151838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.151996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80340) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.152965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.153181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 16:26:07.170051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.170211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 16:26:07.170585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.170731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.170867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 16:26:07.170941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 16:26:07.171088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 16:26:07.171526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.171926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.172074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.172362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.182577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.182849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.183148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.183428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.183569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.183867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.184131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.184413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.191612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.192716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.195039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.195342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.195714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.196021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.196470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.196603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.196682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.196816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.196935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.197985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.198203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.198449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.198743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.199015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.199278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.224635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.226229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.226496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.226620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.228637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.228923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.231773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.232162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.232411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.234285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.234564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.234957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.239806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.240089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.242263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.243193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.244697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.245297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.245574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.245849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.246115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.246380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.246642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.246903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.248036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.248271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.248573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.248835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.249114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.249387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.249655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.249915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.265684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.265968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.268902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.269178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.270523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.274223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.274346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.274615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.277899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.278291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.279622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.283422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.283548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.286988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.287150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.287472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.287752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.288025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.288293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.288566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.291227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.291525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.291807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.292084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.292360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.292631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.309347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.309624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.309883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.310147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.314019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.314304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.314587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.314846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.318088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.319828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.320114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.322983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.323735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.324023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.324306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.324595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.324718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.324847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.324967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.328206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.328489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.328777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.329080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.329368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.338148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.338421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.338709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.338985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.339380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.339655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.339927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.343333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.343612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.343877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.344160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.344439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.344806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.345087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.358057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.361245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.363688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.364605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.366328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.366944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.368173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.368491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.368777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.371356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.373125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.374768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.375074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.375358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.376459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.378102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.381337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.381623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.382007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.382281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.382786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.384984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.388251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.388544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.388936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.389230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.391410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.391695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.391974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.398482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.399288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.399561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.399867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80340) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.400141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.400415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.400704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.400985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.401356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.401630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.401886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.402162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.402428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.402691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.403027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.403319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.403602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80340) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.407499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 16:26:07.407779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.408058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.408333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.408627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.409019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.409297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.409568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.409850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.410126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.410408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.410687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.410961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.411245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 16:26:07.441413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555973c4f9c0 pid: 80340 (default app) Jun 16 16:26:07.444842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835cd5-93be-405a-a2dd-71f6d6573ed8 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:07.445141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835cd5-93be-405a-a2dd-71f6d6573ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:07.445420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835cd5-93be-405a-a2dd-71f6d6573ed8 None None] PATH_INFO: `/v3` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:07.449796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1/1] 10.222.0.74 () {50 vars in 791 bytes} [Thu Jun 16 16:26:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:07.450399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555973c4f9c0 pid: 80339 (default app) Jun 16 16:26:15.313789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:15.314372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:15.314372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:15.316495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 2/2] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:15.325869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:15.326335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:15.326429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:15.396775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: INFO dbcounter [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] Registered counter for database keystone Jun 16 16:26:15.421649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG oslo_db.sqlalchemy.engines [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80339) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 16:26:15.432290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writer thread running {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 16:26:15.613631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80339) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 16:26:15.613946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80339) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 16:26:15.670098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:15.671752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:15.817487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1/3] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 16:26:15.978683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:16.052245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: INFO dbcounter [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] Registered counter for database keystone Jun 16 16:26:16.073946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG oslo_db.sqlalchemy.engines [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80340) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 16:26:16.084643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writer thread running {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 16:26:16.203739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf12ba-2311-4366-a13f-cda41e36fcaa None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:16.208477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:16.209145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:16.209432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:16.209710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:16.218514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:16.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 16:26:16.270121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c8905-832b-46b2-8f9e-9f47261fc0d8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:16.280769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 3/4] 10.222.0.74 () {58 vars in 1324 bytes} [Thu Jun 16 16:26:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:26:16.286994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:16.302073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b850299-3768-4a7d-9c0e-7c1065c788d3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:16.307276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:16.308426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:16.308718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:16.309000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] PATH_INFO: `/v3/projects/f0df30bd71bf4670afbc831b314be545` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:16.310542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] RBAC: Authorizing `identity:get_project(project_id=f0df30bd71bf4670afbc831b314be545)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:16.360294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 16:26:16.360952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e7e6b-6229-4f51-9823-c93a37fdce1f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:16.365836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 2/5] 10.222.0.74 () {56 vars in 1207 bytes} [Thu Jun 16 16:26:16 2022] GET /identity/v3/projects/f0df30bd71bf4670afbc831b314be545 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:18.472507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc65cb-9743-4ad9-b41e-e4a2acb8c42d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:18.473080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc65cb-9743-4ad9-b41e-e4a2acb8c42d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:18.473080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc65cb-9743-4ad9-b41e-e4a2acb8c42d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:18.477898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 4/6] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:18.483689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:18.484155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:18.484248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:18.506124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:18.507783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:18.517435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 3/7] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 16:26:18.540363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc65cb-9743-4ad9-b41e-e4a2acb8c42d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:18.555806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc65cb-9743-4ad9-b41e-e4a2acb8c42d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:18.561158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:18.561892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:18.562181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:18.562456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:18.571396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:18.573198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2b1bf-da4c-4542-96f1-4196240c7057 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:18.582583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 5/8] 10.222.0.74 () {58 vars in 1324 bytes} [Thu Jun 16 16:26:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:26:18.588911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:18.598600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972684a-59ad-4684-90cc-e26898c53f12 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:18.607210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:18.608109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:18.608395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:18.608675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:18.610243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:18.612169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6275dd1-c6e7-4b44-a477-88b74b89a813 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:18.613857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 4/9] 10.222.0.74 () {56 vars in 1198 bytes} [Thu Jun 16 16:26:18 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:20.807099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0216d-4738-4c08-8de8-54486b5e2dd5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:20.807621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0216d-4738-4c08-8de8-54486b5e2dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:20.807621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0216d-4738-4c08-8de8-54486b5e2dd5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:20.808968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 6/10] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:20.814557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:20.814984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:20.815152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:20.822211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582835-5487-4065-8530-c33c1e160e53 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:20.822612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582835-5487-4065-8530-c33c1e160e53 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:20.823099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582835-5487-4065-8530-c33c1e160e53 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:20.825000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 7/11] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:26:20.833112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:20.833472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:20.833732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:20.842893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:20.844606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:20.880100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG passlib.handlers.bcrypt [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80340) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 16:26:20.880522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG passlib.handlers.bcrypt [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80340) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 16:26:20.925800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:20.927460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:20.953228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 5/12] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:20.977530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.002834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.047816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 8/13] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:21.069827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.091229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea9537-3cf9-4719-9bc5-7c3099506dd2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.130675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056434-2c69-496c-b527-5741ea402091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.131522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056434-2c69-496c-b527-5741ea402091 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.131806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056434-2c69-496c-b527-5741ea402091 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.132094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056434-2c69-496c-b527-5741ea402091 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.144271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16056434-2c69-496c-b527-5741ea402091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-16056434-2c69-496c-b527-5741ea402091 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:21.155500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:21.156923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:21.158237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 6/14] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:26:21.163191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056434-2c69-496c-b527-5741ea402091 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.179012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b015603-8a29-403e-9eac-87a3caa90f4f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.181478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.224755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.225662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.225953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.226242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.255300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.274930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056434-2c69-496c-b527-5741ea402091 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.306298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.307890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.314875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.315606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.315898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.316199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.317435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:21.317967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.320604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.335590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 7/15] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:21.341485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.422438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 9/16] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:21.428061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.431246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.457328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969d12-9051-4fbd-8b3b-7d4a5b999251 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.496480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.497968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.498470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.498952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.501068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.503813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.506925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 8/17] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:21.513126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.527996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.537818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0612fc-250c-457b-a97a-48fed3c9284c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.574090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.574820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.575168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.575457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.594988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.596544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:26:21.618265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:21.619278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:21.620682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:26:21.620682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:26:21.620682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:26:21.620682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:21.620682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 10/18] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:21.622440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a16816-6429-4904-a53d-ee2774d08916 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.625681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.659651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.660472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.660759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.661028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.662402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.664645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.666281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 9/19] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:21.674666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:21.691063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.736647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d2842-1057-4aff-90f1-b0406def41a8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.757825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:21.771631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.772308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.772598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.772876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.773636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:21.774130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.775580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff3079-703a-42a8-97de-d361945fa3da None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.780842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304098ba-c699-4bef-8685-e80cc8a642f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:21.794177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 11/20] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:21.832524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81440371-959e-417a-b636-f75c6158c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:21.833098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81440371-959e-417a-b636-f75c6158c502 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:21.833386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81440371-959e-417a-b636-f75c6158c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:21.833665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81440371-959e-417a-b636-f75c6158c502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:21.834530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81440371-959e-417a-b636-f75c6158c502 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.domain.id=default, user.id=55610df78e6e43caada373cf55f4eeb4) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:21.834827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81440371-959e-417a-b636-f75c6158c502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:21.836425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81440371-959e-417a-b636-f75c6158c502 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:21.844326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 10/21] 10.222.0.74 () {56 vars in 1322 bytes} [Thu Jun 16 16:26:21 2022] GET /identity/v3/role_assignments?user.id=55610df78e6e43caada373cf55f4eeb4&scope.domain.id=default&role.id=279eea356bbd45f39595ccaa1656385a => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:24.096543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835900-19c0-4aa6-80c5-71594066513f None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.097220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835900-19c0-4aa6-80c5-71594066513f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.097220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2835900-19c0-4aa6-80c5-71594066513f None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.098726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 12/22] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:24.104107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.104558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.104652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.128332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:24.129580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:24.223576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 11/23] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:24.257855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2835900-19c0-4aa6-80c5-71594066513f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:24.394225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2835900-19c0-4aa6-80c5-71594066513f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:24.412459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:24.440418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:24.441243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.441528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.441802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.460211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:24.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:24.470701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:24.471487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:24.473910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 13/24] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:24.479959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:24.597014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:24.632154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705e19e-0f55-4d08-92a6-8bfe003950b9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:24.696581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:24.697305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.697600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.697885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.698684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:24.698992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:24.700539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dd27c-2af0-468f-bfa9-d02fce021919 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:24.711019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 12/25] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:24.716126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:24.769823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79eae4-b2f4-460d-a421-9a89cebd470b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.771233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79eae4-b2f4-460d-a421-9a89cebd470b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.771731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79eae4-b2f4-460d-a421-9a89cebd470b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.776769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 13/26] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:24.782225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7e1c-0ac2-497a-92f5-9ba44f047ab0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.782759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7e1c-0ac2-497a-92f5-9ba44f047ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.783253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7e1c-0ac2-497a-92f5-9ba44f047ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.816249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-13be7e1c-0ac2-497a-92f5-9ba44f047ab0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:24.818533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13be7e1c-0ac2-497a-92f5-9ba44f047ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:24.894468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f2f7-e609-484f-b6f2-1f6699a514e8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:24.939490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 14/27] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:24.942217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623a33-4dd6-4457-ad8e-2ab8f5e5e2a6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.944317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623a33-4dd6-4457-ad8e-2ab8f5e5e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.945042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623a33-4dd6-4457-ad8e-2ab8f5e5e2a6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.951041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 15/28] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:24.952486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.952983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.953466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.990178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:24.991769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:24.992632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:24.993241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:24.993721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:24.994239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:24.996776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:24.998631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.001299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 14/29] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:24 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:25.003571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.073310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.095375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.105936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 16/30] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:25.107359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.137209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a2a8f-d7db-4ac8-b57f-12855dbc284a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.174882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.175811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.176164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.176492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.177294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.178934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.227084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f9dc9-c837-45f5-8411-62c5146dd0d5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.262742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.263594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.263882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.264160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.265354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.266819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00173089-ebdb-4566-8614-df402b9a41d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.273590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 17/31] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:25.274930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122227-3145-462c-99a0-ddb5adf86a19 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.275245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122227-3145-462c-99a0-ddb5adf86a19 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.275532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122227-3145-462c-99a0-ddb5adf86a19 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.276808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 18/32] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:25.277499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122227-3145-462c-99a0-ddb5adf86a19 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.328878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 15/33] 10.222.0.74 () {60 vars in 1131 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/roles => generated 225 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:25.330373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.355422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.393387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122227-3145-462c-99a0-ddb5adf86a19 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.434677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.435551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.435835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.436114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.449208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.450847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-312bf54c-7782-4234-a2f8-dc940609daf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:26:25.479749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:25.480577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:25.481354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:26:25.481354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:26:25.481354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:26:25.481354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:25.481354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 19/34] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:25.482359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.482639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.482918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.517400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:25.518752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:25.608344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d920a-b843-466e-ae9b-98e70369259f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.612838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.612983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 20/35] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:25.613617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.652955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.654272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.654725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.655235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] PATH_INFO: `/v3/domains/default/users/55610df78e6e43caada373cf55f4eeb4/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.657776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=55610df78e6e43caada373cf55f4eeb4, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.660247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.673922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.682238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 16/36] 10.222.0.74 () {58 vars in 1361 bytes} [Thu Jun 16 16:26:25 2022] PUT /identity/v3/domains/default/users/55610df78e6e43caada373cf55f4eeb4/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:25.683505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.756800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81945c-f4ce-449a-8f78-c9e8cdaca704 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.799472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.800155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.800443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.800726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.803844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:25.804290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.806346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.807591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11479b7-eec2-44e3-b1d1-64259964a55d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.828870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 21/37] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:25.838382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.854349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.855444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.855938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.856523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.857618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.859568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a8b3d-9b26-4a8d-b282-022339bc4d4a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.883495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 17/38] 10.222.0.74 () {60 vars in 1131 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/roles => generated 219 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:25.911607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22ad-cf94-4bd7-a40d-1065df97f56e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.911976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22ad-cf94-4bd7-a40d-1065df97f56e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.912193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e22ad-cf94-4bd7-a40d-1065df97f56e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.914168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 18/39] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:26:25.920743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cba7f-6d3a-4c08-ad28-2a66bb6e7a26 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.921213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cba7f-6d3a-4c08-ad28-2a66bb6e7a26 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.921570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cba7f-6d3a-4c08-ad28-2a66bb6e7a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.063163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e88cba7f-6d3a-4c08-ad28-2a66bb6e7a26 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:26.064736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e88cba7f-6d3a-4c08-ad28-2a66bb6e7a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:26.086045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9c7c-4e30-4776-9edd-5a559491f7aa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:26.097113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:26.115473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:26.142472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:26.143213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.143513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.143796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.145090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:26.146584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:26.161143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 19/40] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:26.162520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdcaf4-d258-4842-ba22-6bcdcee9c738 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.162816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdcaf4-d258-4842-ba22-6bcdcee9c738 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.163126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdcaf4-d258-4842-ba22-6bcdcee9c738 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.165361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 20/41] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:26.170893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.171225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.171517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.191849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:26.193049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:26.271984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 21/42] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:26.273312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:26.306536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 22/43] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/projects => generated 301 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:26:26.307515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:26.375158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:26.395755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29659cb-2555-45d5-afea-7a1ca3560a6b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:26.423638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438a996-b13a-45b4-bb2a-c71b30eefd8c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:26.439870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:26.440573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.440863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.441145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.441923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:26.443687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:26.461970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 22/44] 10.222.0.74 () {60 vars in 1131 bytes} [Thu Jun 16 16:26:26 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:26.464228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:26.465124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.465413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.465741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.467286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:26.469036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58156f5f-9067-437e-b4e0-f98fef079cac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:26.470944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 23/45] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:26.477457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:26.611538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171202d5-fb2c-4a45-a6eb-a528622dda76 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:26.650089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:26.650700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:26.650992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:26.651305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:26.652027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:26.653589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5475b8-a080-41c0-bb95-3d45d8e9e154 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:26.671818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:26.688975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 23/46] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:26:26 2022] POST /identity/v3/projects => generated 312 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:28.052501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251acac-33bd-43dd-9186-debbe2349fa7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.053040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251acac-33bd-43dd-9186-debbe2349fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.053040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251acac-33bd-43dd-9186-debbe2349fa7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.054505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 24/47] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:28.059890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.060311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.060534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.078189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:28.079436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.150787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 24/48] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:28.173548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251acac-33bd-43dd-9186-debbe2349fa7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.204592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.283525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251acac-33bd-43dd-9186-debbe2349fa7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.319375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.320118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.320426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.320707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.332260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:28.340095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:28.341062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:28.341932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 25/49] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:28.348683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.451404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba4d3b-e15b-4698-a4a0-14fe3eb5a1e1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.478310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.487805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.488484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.488773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.489048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.489805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:28.490115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.491553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.503509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 25/50] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:28.509106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.602297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.618370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26fd1-f9bb-4d89-9791-477bac03a333 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.652890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.653780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.654069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.654348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.655682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.657142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b52eb5-2985-4c8d-816e-3b76bae5928a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.658789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 26/51] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:28.664395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.777321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80761fe4-feea-47be-a1c8-b66106cb700f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.806526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0ab9-c0aa-45c5-a9b4-6251037676bd None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.806792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0ab9-c0aa-45c5-a9b4-6251037676bd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.807117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae0ab9-c0aa-45c5-a9b4-6251037676bd None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.808492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 27/52] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:28.813481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.813773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.814056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.814316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.815160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.815452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.815730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.816929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.818390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.820043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 26/53] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:28.826635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.831862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:28.833406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.889848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.907111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 28/54] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:28.927284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.938047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f140c-4403-4def-83f5-3222b65edec4 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.972069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.972736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.973024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.973307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.974144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.domain.id=default, user.id=55610df78e6e43caada373cf55f4eeb4) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:28.974459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.976096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.984763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 27/55] 10.222.0.74 () {56 vars in 1322 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/role_assignments?user.id=55610df78e6e43caada373cf55f4eeb4&scope.domain.id=default&role.id=279eea356bbd45f39595ccaa1656385a => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:29.020049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:29.038514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff596bf-23cc-4fd3-a690-7f69d826faf0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:29.078289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:29.079117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:29.079423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:29.079707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:29.080996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:29.082491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b3f77-5383-469c-bfaf-4a377fcf8916 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:29.084184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 29/56] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:29.090183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:29.199576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d780a-ad0d-4dd1-bb53-9272e602df9a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:29.220038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:29.234924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:29.235650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:29.235927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:29.236211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:29.236938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:29.238419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06ec4e-28d4-43f9-8b9c-f205d84c03dc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:29.272968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 28/57] 10.222.0.74 () {60 vars in 1140 bytes} [Thu Jun 16 16:26:29 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:31.399715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550202e-12d1-40ab-b441-ffe131282bec None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:31.400342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550202e-12d1-40ab-b441-ffe131282bec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:31.400342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550202e-12d1-40ab-b441-ffe131282bec None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:31.401707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 30/58] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:31.408151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:31.408845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:31.408987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:31.429732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:31.430966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:31.509278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 29/59] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:31.544525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550202e-12d1-40ab-b441-ffe131282bec None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:31.655004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550202e-12d1-40ab-b441-ffe131282bec None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:31.693303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:31.694109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:31.694398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:31.694711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:31.696027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:31.697510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264d887-7c4a-46fa-8953-817cca3fbab1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:31.699146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 31/60] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:31.705666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:31.816478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b84df-7cec-4804-94d6-3afe0d09f5a8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:31.818010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:31.862259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:31.862937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:31.863259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:31.863546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:31.864263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:31.865743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb873f3-7391-4fc1-9b84-5e22a827e1f6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:31.894153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 30/61] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:31 2022] POST /identity/v3/users => generated 275 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:35.087653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a874f9-125a-4c85-bbd2-442ef308659e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.089075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a874f9-125a-4c85-bbd2-442ef308659e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.089845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a874f9-125a-4c85-bbd2-442ef308659e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.093067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 32/62] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:35.099243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b856-0599-422a-9066-462db47e4660 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.100581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b856-0599-422a-9066-462db47e4660 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.101412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924b856-0599-422a-9066-462db47e4660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.135376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9924b856-0599-422a-9066-462db47e4660 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:35.137811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9924b856-0599-422a-9066-462db47e4660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:35.240418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 31/63] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:35.275828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a874f9-125a-4c85-bbd2-442ef308659e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:35.336480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:35.479890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a874f9-125a-4c85-bbd2-442ef308659e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:35.538235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:35.539255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.539443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.539720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.568290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a21d1eec-11a9-4250-8712-138aa6db3157 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:35.577201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:35.578673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:35.579996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 33/64] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:26:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:35.588266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924b856-0599-422a-9066-462db47e4660 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:35.621941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868478d8-27f9-4964-88ab-f338e228ace5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.622361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868478d8-27f9-4964-88ab-f338e228ace5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.622598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868478d8-27f9-4964-88ab-f338e228ace5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.624105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 34/65] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:35.629245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351187b-e7a7-460d-b1b8-41831e444ec6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.629781 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351187b-e7a7-460d-b1b8-41831e444ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.630270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351187b-e7a7-460d-b1b8-41831e444ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.668107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5351187b-e7a7-460d-b1b8-41831e444ec6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:35.669578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5351187b-e7a7-460d-b1b8-41831e444ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:35.759601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:35.775950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924b856-0599-422a-9066-462db47e4660 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:35.780406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 35/66] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:35.782513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7b4b-c40c-4e8e-acc6-c29942970d5d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.782972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7b4b-c40c-4e8e-acc6-c29942970d5d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.783630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7b4b-c40c-4e8e-acc6-c29942970d5d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.785121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 36/67] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:35.791820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.792402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.792853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.827568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:35.829074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:35.833326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:35.846101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:35.847556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:35.850312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:35.851271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:35.852833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:35.853304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:35.854943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:35.867545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 32/68] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:26:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:35.868788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:35.918475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 37/69] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:35.920160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.013451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a0ea6-9075-419c-9d8e-1521ba7e0c1f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.036210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474abc3-a2ad-4176-8b4e-11acecac6eab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.059603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.061473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.062246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.062976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.079346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:36.083347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.084370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.084757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.085317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:36.089831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:36.090728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:36.091490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:36.099814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 33/70] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:36.100507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.102721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.106943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.110046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.111512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 38/71] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:35 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.112406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.122356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.231287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec729ecd-b768-43f0-96c7-4538e129877f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.253651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14284fa8-926c-4224-a707-172d6f97b55a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.273528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.274273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.274566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.274858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.287486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:36.297016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:36.297182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:36.297997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:36.298712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:36.299306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.299370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 34/72] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:36.299430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.299430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.299547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.300287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:36.300287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.301436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5701aa1-0859-44f8-90ed-013416f9a2dd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.312894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 39/73] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.314242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4553b5-edcc-4123-8f43-b9f2fa0c78e1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.314542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4553b5-edcc-4123-8f43-b9f2fa0c78e1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.314808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4553b5-edcc-4123-8f43-b9f2fa0c78e1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.316256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 40/74] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:36.318512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3a620-f730-491a-abe8-94608ba65cd4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.318807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3a620-f730-491a-abe8-94608ba65cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.319083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3a620-f730-491a-abe8-94608ba65cd4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.320715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 41/75] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:36.321113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3a620-f730-491a-abe8-94608ba65cd4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.386517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.392875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.420709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93e23f-da6f-4175-b561-68b6b053b02e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.446193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3a620-f730-491a-abe8-94608ba65cd4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.457068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.457721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.458020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.458299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.486653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.487737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.488011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.488291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.489067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:36.489381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.490808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.501803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 42/76] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.502683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.541564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.542984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135f844-db17-43c1-bfa2-e6b9edf83a99 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.621475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.631993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559c544-d498-4eba-ad47-a087e77042e9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.658000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.661892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 35/77] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:36.663420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.663713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.663993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.682141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:36.682355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.683210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.683663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.683807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:36.683961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.687023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.688606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c590d-cfb1-41d6-a0f6-5cb6dea880c3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.690309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 43/78] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.691631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002835e8-c509-4690-916f-a545606f3d77 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.691926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002835e8-c509-4690-916f-a545606f3d77 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.692165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002835e8-c509-4690-916f-a545606f3d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.709472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-002835e8-c509-4690-916f-a545606f3d77 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:36.710689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-002835e8-c509-4690-916f-a545606f3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:36.756346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 36/79] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:36.757263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.789982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 44/80] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:36.790829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002835e8-c509-4690-916f-a545606f3d77 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.860247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.863266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdf07d-2084-4732-ae58-78145f42dbd7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.892466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002835e8-c509-4690-916f-a545606f3d77 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:36.898605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.899590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.899884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.900132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.901348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.902802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.908873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 37/81] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.908873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:36.914444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:36.929827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:36.930707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:36.930992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:36.931299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:36.932508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:36.933962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:36.935709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 45/82] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:36.936481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.013360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a69b9-96ee-455e-91bd-0df0fbff5bbd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.039474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab426-a7aa-4328-b6be-7978436b66b6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.046769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.047467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.047754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.048035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.073032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.073849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.074135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.074413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.082481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.085049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:37.092918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:37.093922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:37.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 46/83] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:37.095696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.119903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.121340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.144001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.195519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa958a88-86bc-4784-b631-acefa170040c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.222916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 38/84] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:37.223827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.231154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.232149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.232436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.232712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.233887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.235392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.236989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 47/85] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.241753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.292895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.326023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721570ef-2208-43c0-833f-98b84b64dfa6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.340560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872ff0c-1843-4ef7-ba19-697862503b05 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.357555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.367744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.368490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.368853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.369129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.376103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.376701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.376990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.377270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.378122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:37.378439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.379967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.385654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 48/86] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=4169efc51b3041918162437c5960c781 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.388480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.449848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.451970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.504669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef59c0-a0ab-4a4c-9005-b9a342525c47 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.540724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.548549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.548752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.549012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.549289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.550091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:37.550404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.551825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.562371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 49/87] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.566045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.587976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 39/88] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:37.593424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.611167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.683908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f605a2-fa58-426e-b3af-cb6b6690dd22 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.693615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e4ff2-3c87-4ef7-9a10-8e765f0be3cf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.721535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.722422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.722708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.722989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.724457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.726064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.727958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 50/89] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.732895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.734373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.735086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.735396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.735644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.736416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.737882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.771084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 40/90] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:37 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:37.771084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.806787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.875637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b419b1-045c-4acd-bcc9-8dc93a32ad6b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.888565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:37.902743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da4af2-2d6d-420a-917f-0c3f68db803a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:37.916122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.917170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.917534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.917885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.919476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.921205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.923225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 51/91] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.924272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:37.942911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:37.944222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:37.945114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:37.945593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:37.947355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:37.949081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:37.951517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 41/92] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:37.952848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:38.056894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384747bc-8537-483f-ad30-64d94f7afd50 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:38.064796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:38.072334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7750aaa-ab67-406f-a4d0-c3b19f9db8c4 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:38.103556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:38.104483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:38.104778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:38.105499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:38.106361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=55610df78e6e43caada373cf55f4eeb4) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:38.106698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:38.108820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:38.113755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:38.114431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:38.114721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:38.115118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:38.117457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 52/93] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/role_assignments?user.id=55610df78e6e43caada373cf55f4eeb4&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:38.118444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:38.199028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:38.226060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:38.227832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:38.278855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34d754-e94d-42ee-9d46-6704df8e3844 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:38.320043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:38.320732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:38.321020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:38.321301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:38.322155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:38.322480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:38.324036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:38.332474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 53/94] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:38 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:38.354591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:38.366845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 42/95] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:38.373463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:38.497739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab242b-ef31-4cad-af38-0d43d51f0165 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:38.505478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:38.535686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:38.536582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:38.536869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:38.537149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] PATH_INFO: `/v3/projects/558f94488cf84b829d002d8cf2ae0c1e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:38.538425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] RBAC: Authorizing `identity:get_project(project_id=558f94488cf84b829d002d8cf2ae0c1e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:38.540036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5055d-f2de-4d91-8101-18e1f83d8899 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:38.541955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 54/96] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:38 2022] GET /identity/v3/projects/558f94488cf84b829d002d8cf2ae0c1e => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:38.548165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:38.670220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64bef-91ce-4a48-9192-5da042c78df1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:38.705692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:38.706371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:38.706663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:38.706947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:38.707830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=558f94488cf84b829d002d8cf2ae0c1e, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:38.708149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:38.709628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b6fa4-f928-40fc-ba0d-c42eb6a571a7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:38.716280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 43/97] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:38 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=558f94488cf84b829d002d8cf2ae0c1e&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:39.882995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b6b60-1f74-4466-b92c-b7cfd0fbe641 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:39.884018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b6b60-1f74-4466-b92c-b7cfd0fbe641 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:39.884505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b6b60-1f74-4466-b92c-b7cfd0fbe641 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:39.886110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 55/98] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:39.891740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:39.892192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:39.892369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:39.914263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:39.915520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:39.983708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:40.009624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 44/99] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:40.048558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b6b60-1f74-4466-b92c-b7cfd0fbe641 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:40.181963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b6b60-1f74-4466-b92c-b7cfd0fbe641 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:40.226491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:40.227755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:40.228237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:40.228707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:40.247926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:40.256616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:40.257612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:40.258496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:40.260470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 56/100] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:26:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:40.266576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:40.403976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e93ad-1d01-4f42-bba6-3909e0a9f964 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:40.448896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:40.449506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:40.449798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:40.450083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:40.450856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:40.451198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:40.452601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:40.461812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 45/101] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:26:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:40.467618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:40.526678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:40.622620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ff07-d64c-430d-b8c3-1ed7093e03f1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:40.671041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:40.671951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:40.672240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:40.672519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:40.679797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:40.681269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:40.682947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 57/102] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:40 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:40.688706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:40.775549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:40.866815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d4d42-cc1b-4f5b-9747-297c8d0a44f1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:40.911750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:40.912633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:40.912921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:40.913256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:40.919816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:40.921456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91466e-66db-412c-b302-417a963a78ed None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:40.923278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 46/103] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:40 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:40.929505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:41.051945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d183ef-e0de-4a1c-a253-7a23b6204753 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:41.053573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:41.090876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:41.091824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.092112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.092392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/f7faeca25d31483582cb85013fa9b8ca/roles/4169efc51b3041918162437c5960c781` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.094714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, user_id=f7faeca25d31483582cb85013fa9b8ca, role_id=4169efc51b3041918162437c5960c781)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:41.096298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0aac3-a8b1-4e5a-9f24-e78a6988c689 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:41.116426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 58/104] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:40 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/f7faeca25d31483582cb85013fa9b8ca/roles/4169efc51b3041918162437c5960c781 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:41.165572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de1b9-e8ea-44c6-a5bc-52441df4c166 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.165914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de1b9-e8ea-44c6-a5bc-52441df4c166 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.166140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de1b9-e8ea-44c6-a5bc-52441df4c166 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.167685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 47/105] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:41.173551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73143ce-ebe5-4919-a3da-320441af7f6f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.173850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73143ce-ebe5-4919-a3da-320441af7f6f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.174131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73143ce-ebe5-4919-a3da-320441af7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.194041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d73143ce-ebe5-4919-a3da-320441af7f6f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:41.196080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d73143ce-ebe5-4919-a3da-320441af7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:41.283712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 59/106] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:41.333364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de1b9-e8ea-44c6-a5bc-52441df4c166 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:41.388328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed48-bbb4-4eb2-98a1-1e66e289bc27 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.388988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed48-bbb4-4eb2-98a1-1e66e289bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.389512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24ed48-bbb4-4eb2-98a1-1e66e289bc27 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.391227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 60/107] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:26:41.397718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.398484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.399006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.421592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:41.423208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:41.513786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de1b9-e8ea-44c6-a5bc-52441df4c166 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:41.563178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:41.563964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 61/108] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:41.585278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:41.586144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.586439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.586724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.588319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:41.590098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:41.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 48/109] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:41.598006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:41.623086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:41.646370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:41.799362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43263-65f4-4343-b52b-da457a750a1a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:41.817990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d754c01-c0d5-426d-96f7-ff7b67dfff88 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:41.868642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:41.869236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.869827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.870099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.870827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:41.872336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0b812-04a9-44ca-bc1f-590e32b79e7a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:41.883757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:41.884582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.884871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.885151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.902386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-a2d8dd5d-9df7-4fb9-9cb2-0b0704dc0274 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:41.910628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:41.911512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:41.912701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 49/110] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:26:41.913596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28ba3-1a08-47ab-8de2-1c94142efcb5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.913761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28ba3-1a08-47ab-8de2-1c94142efcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.914180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28ba3-1a08-47ab-8de2-1c94142efcb5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.920085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 50/111] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:41.920960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28ba3-1a08-47ab-8de2-1c94142efcb5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:41.932537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 62/112] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:26:41 2022] POST /identity/v3/projects => generated 302 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:41.933991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:41.934282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:41.934565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:41.980423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:41.982090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:42.055123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.112329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 63/113] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:42.126350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28ba3-1a08-47ab-8de2-1c94142efcb5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.132617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.140596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.191811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.192516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.193120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.193383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.194186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:42.194815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:42.197194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201006-a255-43ca-90d3-d34b2a307c68 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:42.214218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 51/114] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:42.215677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafae1c-5ae7-418e-9272-55ec15a66744 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.215974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafae1c-5ae7-418e-9272-55ec15a66744 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.216258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafae1c-5ae7-418e-9272-55ec15a66744 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.217755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 52/115] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:42.228146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafae1c-5ae7-418e-9272-55ec15a66744 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.291217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bfffa-cf61-4ba1-bad7-81f961572800 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.329801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.330720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.331009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.331325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.343630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-015cadc7-e1a5-4c79-aaf3-0fd5eae6209d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:42.352218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:42.353185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:42.353930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 64/116] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:26:42.355337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.355337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.355337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.372964 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafae1c-5ae7-418e-9272-55ec15a66744 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.377642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:42.379213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:42.381422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.412192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.413160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.413451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.413773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] PATH_INFO: `/v3/users/279dd73d79644e0a908ecfc22d09412c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.431421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] RBAC: Authorizing `identity:get_user(user_id=279dd73d79644e0a908ecfc22d09412c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:42.432911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7427f6-799a-4caf-b979-a97a0db5e873 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:42.434663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 53/117] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/users/279dd73d79644e0a908ecfc22d09412c => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:42.440150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c7a6-635c-4862-88bf-476441194914 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.440488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c7a6-635c-4862-88bf-476441194914 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.440882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926c7a6-635c-4862-88bf-476441194914 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.442927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 54/118] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:42.447863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926c7a6-635c-4862-88bf-476441194914 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.456535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.468541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 65/119] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:42.475079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.598132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926c7a6-635c-4862-88bf-476441194914 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.625069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8ff96-8ad7-41f3-8bdb-58b607360e8d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.641272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.642566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.643149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.643703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.644606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:42.645205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:42.647025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786109a1-3c11-45d9-b7a3-637f635803e4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:42.658955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 55/120] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:42.660660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.661379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.662068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.666158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.666935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.667340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.667717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.683085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:42.685032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:42.711897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.734752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:42.744801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:42.746692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:42.773950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 56/121] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:42.780175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.890955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 66/122] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:42.896583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:42.913910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c5c11-5875-4448-8374-c596bdb5a18c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:42.949883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:42.950962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:42.951403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:42.951844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:42.963099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:42.971284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:42.972124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:42.972859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:42.975272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 57/123] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:42.981012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.001030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.001278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.011228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfdb65-f57a-4b89-bdc3-f0e8bb3c5798 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.047311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.048612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.048906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.049184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.050402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.051914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.054120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 67/124] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.054967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.092692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a3b6-0152-406b-98c2-a8809169fe57 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.129463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.130630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.131156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.131656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.142723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:43.150832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:43.151684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:43.152448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:43.154765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 58/125] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:43.155760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.161572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d781e-0bc9-4775-89ba-ff06198583db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.197606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.198485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.198770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.199069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.200339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.201858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.203687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 68/126] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.204456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.240145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.242114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.266970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acbedf-fdae-4b4a-8069-9a6f16a2ac5a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.304350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.305573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.306057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.306526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.307616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:43.308120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.309738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.319411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 59/127] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.324983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.325619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efadbd-0135-4866-b555-ca19b7fde5d6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.367806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.369870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.370362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.370834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] PATH_INFO: `/v3/projects/558f94488cf84b829d002d8cf2ae0c1e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.372846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] RBAC: Authorizing `identity:get_project(project_id=558f94488cf84b829d002d8cf2ae0c1e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.375201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.377728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 69/128] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/projects/558f94488cf84b829d002d8cf2ae0c1e => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.379143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.442175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db0ad-abd5-4e2c-a2db-c674d33a86a0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.481571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.482515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.482809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.483517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.484554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:43.484875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.485519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.488192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.489919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b6718-7fb9-4f13-aa72-3670a79e6003 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.496594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.502160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 60/129] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.504009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.539990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.540670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.540956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.541232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.542079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=279dd73d79644e0a908ecfc22d09412c) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:43.542398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.546246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.550421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 70/130] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/role_assignments?user.id=279dd73d79644e0a908ecfc22d09412c&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.560458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.651652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcb99-585b-45d1-8550-df777337d08a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.700151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.702163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.702685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.703214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.705596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.707866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.710319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 61/131] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.711565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.728324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0767-32c6-4c0d-9f82-7a64d4fc1df4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.777964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.779037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.779462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.779915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] PATH_INFO: `/v3/projects/558f94488cf84b829d002d8cf2ae0c1e/users/f7faeca25d31483582cb85013fa9b8ca/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.782950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] RBAC: Authorizing `identity:create_grant(project_id=558f94488cf84b829d002d8cf2ae0c1e, user_id=f7faeca25d31483582cb85013fa9b8ca, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.785380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d93f1-b4fa-4a67-844f-2baced09f801 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.803287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.808707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 71/132] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:43 2022] PUT /identity/v3/projects/558f94488cf84b829d002d8cf2ae0c1e/users/f7faeca25d31483582cb85013fa9b8ca/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:43.808707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f52d-c313-4ba5-a1bf-fc22292368ef None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.808707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f52d-c313-4ba5-a1bf-fc22292368ef None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.808707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7f52d-c313-4ba5-a1bf-fc22292368ef None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.819146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 72/133] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:43.819146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7f52d-c313-4ba5-a1bf-fc22292368ef None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:43.875440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:43.889828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513279-b402-4e05-9ab3-8ceeb5e7c017 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:43.957853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:43.958683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.958969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.959276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:43.961556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:43.963063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd7c5e-7eb2-4150-adcb-75374861b7c8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:43.967988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 62/134] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:43.969302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:43.969599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:43.969883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.013764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:44.014074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:44.034054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7f52d-c313-4ba5-a1bf-fc22292368ef None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.075127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.076019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.076310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.076592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.077807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:44.079468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:44.081592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 73/135] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:43 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:44.082488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.091096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 63/136] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:44.092386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.178201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:44.206787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:44.218908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b33e7-50d5-421c-9c9e-0a83d28e7bf6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.225812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25c2a0-b635-4252-b447-e97d0c56fbaf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.259083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.260397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.260904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.261401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/f7faeca25d31483582cb85013fa9b8ca/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.263761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, user_id=f7faeca25d31483582cb85013fa9b8ca, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:44.265501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:44.293727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 64/137] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:44 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/f7faeca25d31483582cb85013fa9b8ca/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:44.295659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.298954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.300748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.301078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.301363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.305156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:44.309993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:44.314780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 74/138] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:44 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:44.321650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.483829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fdf00-f39e-4e0f-80d8-94410a1fa942 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.518193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:44.535556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.536635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.537068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.537687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.564155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-c9bfbf32-907c-458f-9b93-edd1ae980ba4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:44.573070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:44.573969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:44.574698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:44.583887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4536a1-0706-4c34-9e68-7cf60b13567c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.585612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 65/139] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:26:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:44.587329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099665d-41eb-4d93-ad0e-f04d075ed466 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.587843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099665d-41eb-4d93-ad0e-f04d075ed466 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.588366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099665d-41eb-4d93-ad0e-f04d075ed466 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.589911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 66/140] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:44.595184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.595690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.596230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.615983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:44.617054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.618013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.618240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:44.618338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.618578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/55610df78e6e43caada373cf55f4eeb4/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.623881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, user_id=55610df78e6e43caada373cf55f4eeb4, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:44.624648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:44.641172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 75/141] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:44 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/55610df78e6e43caada373cf55f4eeb4/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:26:44.642064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.665691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:44.698701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 67/142] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:44.742161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.786159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a107d-7161-4321-b972-f3ebfd9f840e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.866098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:44.866825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:44.867116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:44.867404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:44.868179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:44.868491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:44.871917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:44.881865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 76/143] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:26:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:44.887454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:44.969588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b54d3-0a24-4ccc-9cbd-fd2edcf7b5ec None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:44.982324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:45.004898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d92c0-f952-4219-a973-8c539a24383f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:45.007015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:45.045692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.046590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.046875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.047184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.048416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.049961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.051747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 77/144] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:44 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:45.059094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:45.059789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.060624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.060910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.061187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.062410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.063923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.065556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 68/145] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:45.071068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:45.168990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ede86c-c2ae-491a-9a49-de8baebd05c5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:45.205621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.206285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.206626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.207006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.228907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:45.283843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.285105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cfcb7-a600-43c7-9125-cf7371c64003 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:45.285284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.345483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.346532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.347062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.347536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.348444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.350111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.384123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 69/146] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:45 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:26:45.397100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 78/147] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:26:45.403246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:45.453197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:45.537527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdbd32-a45e-4e38-874f-52febbbca2f2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:45.590507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.591435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.591725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.592000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.593271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.594731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f87e0b-1f57-4753-8a89-92409b93cb49 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.596525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 70/148] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:45 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:45.603462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:45.695437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:45.764232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44ea63-b9dc-4624-b84e-cd6fde015ffa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:45.815717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:45.816314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:45.816602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:45.816884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:45.818266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:45.818588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:45.820168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69084d-c692-4623-b0d1-f9591d444864 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:45.826867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 79/149] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:45 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=4169efc51b3041918162437c5960c781 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:46.367483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea211f-3ce7-4ccf-bfd2-f39c9727a27b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:46.367992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea211f-3ce7-4ccf-bfd2-f39c9727a27b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:46.368062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea211f-3ce7-4ccf-bfd2-f39c9727a27b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:46.369762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 71/150] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:46.375310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:46.375719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:46.375870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:46.395498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:46.396708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:46.472972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 80/151] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:46.497450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea211f-3ce7-4ccf-bfd2-f39c9727a27b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:46.667615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea211f-3ce7-4ccf-bfd2-f39c9727a27b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:46.677356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:46.708776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:46.709603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:46.710277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:46.710980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:46.722069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-dcd5495d-d58b-4a2c-9b1a-0e84961c6713 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:46.730550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:46.731498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:46.732232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 72/152] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:46.739505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:46.740688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81043-dbd8-455a-b5c7-f8c79122ee89 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:46.741116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81043-dbd8-455a-b5c7-f8c79122ee89 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:46.742053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf81043-dbd8-455a-b5c7-f8c79122ee89 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:46.743470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 73/153] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:26:46.749430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:46.749968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:46.750432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:46.774272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:46.775816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:46.833716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:46.886824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 74/154] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:46.913834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:46.937297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9f34-2352-4787-a769-bd97641498ce None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:46.999877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.000610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.000893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.001176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.001948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:47.002271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:47.003711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:47.021206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 81/155] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:47.032390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.047250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:47.051325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e08fe-f9a8-47af-9772-b0ae5fc21fd4 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:47.104738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.105540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.105831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.106121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.128846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:47.139428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:47.140510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:47.141986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 75/156] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:26:47.152421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.260134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ccda2-9a58-4107-b589-51fb30d33ad3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:47.268264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:47.314289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866538f3-6d54-4b3c-9aed-ebd5f8b340b0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:47.344258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.345205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.345500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.345775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] PATH_INFO: `/v3/users/279dd73d79644e0a908ecfc22d09412c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.347004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] RBAC: Authorizing `identity:get_user(user_id=279dd73d79644e0a908ecfc22d09412c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:47.348560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:47.350223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 82/157] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:47 2022] GET /identity/v3/users/279dd73d79644e0a908ecfc22d09412c => generated 289 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:47.356025 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.379804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.380503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.380796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.381080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.382537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:47.382864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:47.384325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef3dc6-b285-42e2-b14b-8c293e2118e2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:47.397079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 76/158] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:47.403192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb790-4a0a-44a7-bba3-e0b0740dcbd8 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.403665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb790-4a0a-44a7-bba3-e0b0740dcbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.404158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb790-4a0a-44a7-bba3-e0b0740dcbd8 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.405863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 77/159] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:47.407254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fb790-4a0a-44a7-bba3-e0b0740dcbd8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.460176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:47.575765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad9ea-bd4e-44cf-b41d-4d53aa1259c5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:47.660727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.661883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.662165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.662438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.667730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fb790-4a0a-44a7-bba3-e0b0740dcbd8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:47.667872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:47.669514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13050b19-d8c8-4fdb-8074-2c8d3a9b66c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:47.672216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 83/160] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:47 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:47.673404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.673686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.673962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.704225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:47.705435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:47.749289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:47.750191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:47.750477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:47.750755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:47.756352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:47.758224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:47.759944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 78/161] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:47 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:47.760699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.774074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:47.861105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 84/162] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:47.861883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:47.972296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:48.000606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732b3c-f301-48c0-a1e6-3c2c357cf58e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.070640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.071591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.071879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.072160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/279dd73d79644e0a908ecfc22d09412c/roles/c1d2e0a6311d4b40ad03045bc9d276dc` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.074171 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, user_id=279dd73d79644e0a908ecfc22d09412c, role_id=c1d2e0a6311d4b40ad03045bc9d276dc)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:48.079896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:48.102843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 79/163] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:47 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/users/279dd73d79644e0a908ecfc22d09412c/roles/c1d2e0a6311d4b40ad03045bc9d276dc => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:48.103788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:48.113923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ac090-c7d7-4fe1-aff2-9d5500a61374 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.215960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.216637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.216919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.217191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.245823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce016f-6f24-423e-8bca-bea346eb21a7 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.286890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.289605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.290783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.291281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.311439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:48.316744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-cca270fb-3f92-47b4-8491-8ead87960613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:48.328488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:48.329561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:48.330461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:48.334906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 80/164] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:48.337649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf938c3-ea0d-492e-a11f-d42381be938b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.337942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf938c3-ea0d-492e-a11f-d42381be938b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.338220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf938c3-ea0d-492e-a11f-d42381be938b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.342100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 81/165] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:48.342902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf938c3-ea0d-492e-a11f-d42381be938b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:48.369927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:48.371371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:48.372316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:48.477166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf938c3-ea0d-492e-a11f-d42381be938b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.544948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.545636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.545922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.546197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.546970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:48.547319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:48.548731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851db10-2c4f-4286-88f4-b574820d9ce3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:48.572068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 82/166] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:48.574391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.574680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.574957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.582934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 85/167] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:48.587961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:48.614867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:48.616344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:48.701986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:48.709416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 83/168] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:48.710374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:48.740921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:48.806296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa92891-e2ee-4d1b-9c34-16279e1561fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.867689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.868685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.868975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.869250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] PATH_INFO: `/v3/users/55610df78e6e43caada373cf55f4eeb4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.871869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] RBAC: Authorizing `identity:get_user(user_id=55610df78e6e43caada373cf55f4eeb4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:48.873415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:48.878063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 86/169] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/users/55610df78e6e43caada373cf55f4eeb4 => generated 247 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:48.878878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:48.895407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0428-77d9-4ad4-99ac-d648f9db6c78 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:48.944109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:48.945180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:48.945465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:48.945743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] PATH_INFO: `/v3/projects/558f94488cf84b829d002d8cf2ae0c1e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:48.947584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] RBAC: Authorizing `identity:get_project(project_id=558f94488cf84b829d002d8cf2ae0c1e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:48.949235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:48.951141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 84/170] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/projects/558f94488cf84b829d002d8cf2ae0c1e => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:48.952110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.021995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.056734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134579f-cc05-4946-ae31-ce6c288edfbf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.066973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52892b-c094-47f7-8e4f-f823df2fe00c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.092285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.092942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.093225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.093499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.125299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.126572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.127073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.127527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.145877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.155965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:49.164687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.166346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:49.168755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:49.169797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 87/171] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:49.170835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.238879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.275468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 85/172] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:49.276751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.384619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425ea99-e8cd-4a58-bf5a-c041b476b002 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.422232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.422704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbb61-f203-4b99-9d7f-e966a4321cdf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.423387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.423810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.424325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.425432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:49.425887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.427482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.437149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 86/173] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:49.438290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.491599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.496616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.497493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.497602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.497844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.498697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=558f94488cf84b829d002d8cf2ae0c1e, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:49.498990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.500699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.516461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 88/174] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=558f94488cf84b829d002d8cf2ae0c1e&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:49.517318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.594647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7368d3-4d64-4efc-836c-afdaee532d2c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.651344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.652331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.652617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.652893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.660199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.661713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.664126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 87/175] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:49.665989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.670413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.747710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23372-9237-453a-b36b-726447511bf0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.809256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.810155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.810594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.811004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] PATH_INFO: `/v3/users/f7faeca25d31483582cb85013fa9b8ca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.812432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] RBAC: Authorizing `identity:get_user(user_id=f7faeca25d31483582cb85013fa9b8ca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.814034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac0fd3-ad0c-4378-bbbd-a3e2e299c4cb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.815880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 89/176] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/users/f7faeca25d31483582cb85013fa9b8ca => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:49.823083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01222d2e-ced7-4e13-a7f5-84318679f9d5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.823762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01222d2e-ced7-4e13-a7f5-84318679f9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.824213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01222d2e-ced7-4e13-a7f5-84318679f9d5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.826009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 90/177] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:49.827079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01222d2e-ced7-4e13-a7f5-84318679f9d5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:49.847806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19e73-0202-48e4-ae45-53b14e9d0478 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:49.849579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:49.931416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:49.931416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.931416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.931416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.931874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=55610df78e6e43caada373cf55f4eeb4) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:49.931874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:49.931874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dc51b-c275-4963-907b-2ba9c3a5366f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:49.951254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 88/178] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/role_assignments?user.id=55610df78e6e43caada373cf55f4eeb4&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:49.951254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:49.951254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:49.951254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:49.977351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:49.978565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:49.991904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01222d2e-ced7-4e13-a7f5-84318679f9d5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.029347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.040992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.042034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.042456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.042865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.066512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 89/179] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:50.109544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.157774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:50.159349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15450407-5682-4788-8377-c8aabdc1a59b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:50.309632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.320676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229deed-fa24-44de-b1a5-15606fc6b5fa None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.350687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 91/180] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:50.352446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe24b3b-92bd-4609-a226-9f6b4f6ddb3e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.352747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe24b3b-92bd-4609-a226-9f6b4f6ddb3e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.353032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe24b3b-92bd-4609-a226-9f6b4f6ddb3e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.354729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 92/181] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:50.360526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.360835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.361110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.373334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.374184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.374474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.374753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.383785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:50.385132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:50.389127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:50.397288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:50.398395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:50.399190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 90/182] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:26:50.399821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.421900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.490114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 93/183] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:50.491766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.535840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d284-33c6-4614-8970-36cdef23534d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.573068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.573869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.574154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.574434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.575655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:50.577121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344866cd-f9eb-4828-b4a6-1be574663792 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:50.579125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 91/184] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:50.580205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614a716-ab2a-46ab-b0e9-8c3c414c556f None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.580500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614a716-ab2a-46ab-b0e9-8c3c414c556f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.580780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614a716-ab2a-46ab-b0e9-8c3c414c556f None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.582003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 92/185] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:50.586148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614a716-ab2a-46ab-b0e9-8c3c414c556f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.608705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.614357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7106c-3761-4d90-8aa0-10edb1a07234 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.654598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.655317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.655595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.655863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.656670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:50.656950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:50.658375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:50.672397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 94/186] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:50.672397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.696566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.702780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614a716-ab2a-46ab-b0e9-8c3c414c556f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.740310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.741326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.741614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.741894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.753460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-cd90b4ef-cb54-469b-93e2-1e3653c2ef26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:50.762457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:50.763478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:50.764225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 93/187] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:50.765628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.765628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.765628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.781509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b2e63-699c-43cb-932a-81f44ca4656f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:50.782246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:50.783488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:50.823035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:50.823796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:50.824024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:50.824302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:50.825184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=f7faeca25d31483582cb85013fa9b8ca) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:50.825839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:50.826983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:50.834213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 95/188] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/role_assignments?user.id=f7faeca25d31483582cb85013fa9b8ca&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:50.835635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.873485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 94/189] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:50.880441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:50.906354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.963185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:50.986643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2b7da-dc8b-4eb0-9657-6a0ee9f7e26c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.028893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.029797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.030082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.030361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.030982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a35f-616f-4ace-9093-3ebb86f25df1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.048634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.050437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.053215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 96/190] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.055034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.081607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.082219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.082484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.082760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.083570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:51.083906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.085438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a935-0586-416f-81bb-b99ef36e56c3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.094881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 95/191] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:26:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.096213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc6f4-1d32-4655-9b8c-27734c38ae1d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.096497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc6f4-1d32-4655-9b8c-27734c38ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.096776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc6f4-1d32-4655-9b8c-27734c38ae1d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.103152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 96/192] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:51.104023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fc6f4-1d32-4655-9b8c-27734c38ae1d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.162265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485350-ee8f-4a56-a3c2-3ff9fe52f60e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.174642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.197647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.198465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.198746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.199017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.200224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.201682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.203355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 97/193] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.204186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.212206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fc6f4-1d32-4655-9b8c-27734c38ae1d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.220002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.248720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.249349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.249634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.249917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.308932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5645-f0e1-4bd2-b584-8fa402663728 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.322787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.324334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.343616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.344428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.344710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.344982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.346246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.347724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8788939-a9b8-4f8f-a0d5-b16e7156b773 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.349364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 98/194] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.352375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.352655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.352911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.370636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:51.372028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:51.394929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.427153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 97/195] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:51.427970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.444002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 99/196] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:51.445601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.458527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.532818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7f65-9b8e-4acb-a6b8-765bfbb10d86 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.552919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d26bb6-0d75-4b18-b804-3f659efbf456 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.570635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.571347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.571638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.571912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.572629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.574081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.592539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.593212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.593600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.594078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.599250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 98/197] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:51 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:51.600084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.644780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.680342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.681869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.696780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.727097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d68a-8673-46ab-8751-3dd039ba54e7 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.764182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.765060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.765354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.765616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.767094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.768339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.771256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 99/198] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.771256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.808657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 100/199] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:51.809628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:51.908308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.916825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afcdf32-bdd0-4f67-8c53-29be686349b1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.939414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ad18e-4388-4a59-b842-2cdf4a32e280 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:51.953753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:51.961692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.962701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.963301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.963576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.977091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:51.983198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:51.983868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:51.984147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:51.984420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:51.985252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=ae80f1b2c9324bdeb1e3eb4eec88300c) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:51.985560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:51.987003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:51.993005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:51.993891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:51.994962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:51.995719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 101/200] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/role_assignments?user.id=ae80f1b2c9324bdeb1e3eb4eec88300c&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:51.995796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 100/201] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:51.995860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:52.004170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:52.183387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe5e8-ee96-4055-8784-dac25e1851df None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:52.195070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf350c-2c15-454f-a4e7-e63363d79615 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:52.224672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:52.225465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:52.225931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:52.226389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:52.227205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:52.227521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:52.229094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:52.243080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 101/202] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:52.243266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:52.243266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:52.243623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:52.243825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:52.246096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:52.247182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:52.248696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:52.252034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 102/203] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:51 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:52.258693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:52.271082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:52.339653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:52.395305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8686ee9-d8d2-44df-b156-95f0d22c7d71 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:52.432862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:52.433820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:52.434090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:52.434374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] PATH_INFO: `/v3/users/279dd73d79644e0a908ecfc22d09412c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:52.435636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] RBAC: Authorizing `identity:get_user(user_id=279dd73d79644e0a908ecfc22d09412c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:52.437120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:52.443179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 102/204] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:52 2022] GET /identity/v3/users/279dd73d79644e0a908ecfc22d09412c => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:52.445447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:52.477273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ac46f-5d0e-48b8-a0c9-24da35b281f7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:52.537277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:52.538672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:52.539961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:52.540462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:52.548636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=ae80f1b2c9324bdeb1e3eb4eec88300c) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:52.549512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:52.553005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:52.568293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 103/205] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:52 2022] GET /identity/v3/role_assignments?user.id=ae80f1b2c9324bdeb1e3eb4eec88300c&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=4169efc51b3041918162437c5960c781 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:52.578698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:52.598564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce66e9-234a-46b0-bd13-a623a202ff2a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:52.642363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:52.643041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:52.643353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:52.643633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:52.767235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:52.768655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:52.884996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:52.901129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 103/206] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:26:52.907017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:53.049507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364086c-7aa9-4073-8bdc-dc8abd724ced None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:53.071458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:53.086426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:53.087347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:53.087664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:53.087938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:53.089134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:53.090595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a332ba-ab98-496f-9cf8-436b73f5c68a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:53.092645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 104/207] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:52 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:53.098774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:53.251998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f049be-10be-4bd7-a2df-8704a170add6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:53.302147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:53.302733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:53.303015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:53.303397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:53.304390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d, user.id=279dd73d79644e0a908ecfc22d09412c) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:53.304709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:53.306180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea109f-afa0-490c-8a7f-c40a413a0df1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:53.312765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 104/208] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:53 2022] GET /identity/v3/role_assignments?user.id=279dd73d79644e0a908ecfc22d09412c&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:54.131804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc96a7-ad03-419a-a75b-694e1e8d08f6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.132787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc96a7-ad03-419a-a75b-694e1e8d08f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.132787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc96a7-ad03-419a-a75b-694e1e8d08f6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.133875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 105/209] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:54.139285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dc0c5-61bc-4f7a-af8b-b590220621ef None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.140138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dc0c5-61bc-4f7a-af8b-b590220621ef None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.140621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dc0c5-61bc-4f7a-af8b-b590220621ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.157670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-dc7dc0c5-61bc-4f7a-af8b-b590220621ef None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:54.159223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dc7dc0c5-61bc-4f7a-af8b-b590220621ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:54.210383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.241200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 105/210] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:54.264238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc96a7-ad03-419a-a75b-694e1e8d08f6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.379223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc96a7-ad03-419a-a75b-694e1e8d08f6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:54.414166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:54.414890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.415223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.415499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.419221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1306-d59a-4ca7-8a47-e9d6ad33034c None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.419776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1306-d59a-4ca7-8a47-e9d6ad33034c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.420254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1306-d59a-4ca7-8a47-e9d6ad33034c None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.421888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 106/211] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:26:54.427174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:54.428038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cefab-8a0d-49f1-8096-ec00a3a2d156 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.429308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cefab-8a0d-49f1-8096-ec00a3a2d156 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.429810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cefab-8a0d-49f1-8096-ec00a3a2d156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:54.435538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:54.436575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:54.437618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 106/212] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:54.442717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.455028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6c9cefab-8a0d-49f1-8096-ec00a3a2d156 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:54.457170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6c9cefab-8a0d-49f1-8096-ec00a3a2d156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:54.483232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.551201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 107/213] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:54.553689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeebe40-5597-4518-87ee-27b64e319c53 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.554420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeebe40-5597-4518-87ee-27b64e319c53 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.554913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeebe40-5597-4518-87ee-27b64e319c53 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.557443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 108/214] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:54.562866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.563188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.563466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.567820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9877-c9bc-4ca0-b119-bde0af50a376 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:54.582893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:54.584098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:54.602848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:54.603633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.603916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.604189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.604934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:54.605237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:54.606614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:54.615428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 107/215] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:54.620325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.671178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 109/216] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:54.671178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.721930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.722956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.729869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e79797-74f9-42f1-b82f-6e4a9bdd84e3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:54.770368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:54.771219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.771510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.771788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.773085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:54.774551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:54.780760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 108/217] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:54.780760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.791182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b796de-05e9-46f7-aa36-f77e15bad513 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:54.829393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:54.830291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.830578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.831077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] PATH_INFO: `/v3/users/257fdb9d95884e5ea16dcf1e36a380d7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.846620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] RBAC: Authorizing `identity:get_user(user_id=257fdb9d95884e5ea16dcf1e36a380d7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:54.849165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:54.851718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 110/218] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/users/257fdb9d95884e5ea16dcf1e36a380d7 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:54.856762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.886831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688c137-0738-4e97-a01b-e32bfda9ddf4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:54.921696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:54.922402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:54.922700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:54.922961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:54.933789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:54.941681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:54.942774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 109/219] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:54.947855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:54.978719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.979377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:54.985736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cee78f-2f69-4b7a-9ed6-20c08d871a9a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.024307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.024995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.025278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.025559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.026360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.027858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.048094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 111/220] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:54 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:55.049008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.072833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889c3d7-d275-4e2c-a020-4c79d25f5bc1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.116156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.116936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.117223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.117505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.167465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bc569-901d-461d-a99d-cbb839f81f43 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.205566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.206160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.206869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.207086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddd790-2331-47aa-8f83-00fe02964849 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.207263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.207542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.208554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:55.209067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.210645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede96ae-e755-4179-8516-479bfabca088 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.220454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 112/221] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:55.222070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8593b-5682-4eb9-a6c6-b00fe26df3c6 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.222358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8593b-5682-4eb9-a6c6-b00fe26df3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.222635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8593b-5682-4eb9-a6c6-b00fe26df3c6 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.224060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 113/222] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:55.228016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8593b-5682-4eb9-a6c6-b00fe26df3c6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.245295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.256295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.347647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 110/223] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:55.348751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.349038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.350319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.367657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:55.368849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:55.372166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8593b-5682-4eb9-a6c6-b00fe26df3c6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.415560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.416462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.416744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.417020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.418321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.419827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.421752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 114/224] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:55.422373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.461586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 111/225] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:55.462364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.536448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.540593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.564224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84d39f-b75c-41d8-a56a-ecb0eef69004 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.607429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765ecd8-8710-4038-a6cc-9470614a7f3c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.610963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.611980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.612362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.612739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.614117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.616159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.619011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 115/226] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:55.623953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.659407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.660560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.660941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.661536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.663599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.665806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.668567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 112/227] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:55.671636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.762437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2f74-d2a0-4af2-a26b-b835c7c4733a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.799994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.800659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.800947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.801235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.804116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c709-9e5a-4625-9727-174d3004869e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.819022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:55.831169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:55.831643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:55.832836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:26:55.833797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:55.837101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 116/228] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:55.838521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.851082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.851586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.851871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.852193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.853036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=257fdb9d95884e5ea16dcf1e36a380d7) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:55.853743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.854805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.861302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 113/229] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/role_assignments?user.id=257fdb9d95884e5ea16dcf1e36a380d7&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:55.861647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.862263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.962745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b707e9-6a2b-47ad-aabc-f68c2e7d3167 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.999740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fdeb3-d10d-450b-8cf3-755965647911 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.000570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.001484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.001768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.002044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/ae80f1b2c9324bdeb1e3eb4eec88300c/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.005488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, user_id=ae80f1b2c9324bdeb1e3eb4eec88300c, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.007272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.026559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 117/230] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:55 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/ae80f1b2c9324bdeb1e3eb4eec88300c/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:56.065654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.066914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.067442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.067950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.069061 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:56.069603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.071317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.086345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 114/231] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:56.092847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.192912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:56.244739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4b5f-290e-42d1-b01b-d1790b2e8e08 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.285592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.286418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.286705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.286981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.288379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.289909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.292015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 118/232] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:56.297232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.357162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:56.421241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7fff0-3eff-41f1-9dd0-abfd1db6e47f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.468720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.469609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.469892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.470173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.471416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.472892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58153-60e7-4753-bb26-f60d502585cb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.474935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 115/233] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:56.481358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.650979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acdc16-1342-48e0-a966-53cc03425d03 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.697011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.698302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.698788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.699299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/ae80f1b2c9324bdeb1e3eb4eec88300c/roles/4169efc51b3041918162437c5960c781` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.705322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, user_id=ae80f1b2c9324bdeb1e3eb4eec88300c, role_id=4169efc51b3041918162437c5960c781)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.707027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea7d20-7d06-40ec-8eef-b0aa8a2d01b3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.729053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:56.733988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 119/234] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:56 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/ae80f1b2c9324bdeb1e3eb4eec88300c/roles/4169efc51b3041918162437c5960c781 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:57.652173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9a7b-026e-4550-ad60-6312aa4212c2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:57.652687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9a7b-026e-4550-ad60-6312aa4212c2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:57.652784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c9a7b-026e-4550-ad60-6312aa4212c2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:57.654393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 116/235] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:57.669151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:57.669650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:57.669732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:57.695547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:57.696810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:57.786851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 120/236] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:57.817139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c9a7b-026e-4550-ad60-6312aa4212c2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:57.960941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c9a7b-026e-4550-ad60-6312aa4212c2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:57.978991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:58.012787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.013611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.013889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.014160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.032100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-907e3e4d-13c5-4e34-a0a5-2695d6b80657 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:58.040008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:58.040866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:26:58.041643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 117/237] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:58.048355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.065364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee6333-5483-4603-88cc-77c9c92c00a9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.065709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee6333-5483-4603-88cc-77c9c92c00a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.065922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee6333-5483-4603-88cc-77c9c92c00a9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.067643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 118/238] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:58.072967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.073254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.073533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.092157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:58.093405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:58.190576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 119/239] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:26:58.222994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f6ea6-343a-4d9a-91f9-110664e4ce84 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:58.227513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.288554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:58.301354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.302076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.302359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.302633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.303592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:58.304003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:58.305403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:58.326237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 121/240] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:58.331402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.427537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c5744-583e-4557-bc23-6a36ad2171dc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:58.443183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:58.492956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.494144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.494731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.495014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.496521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:58.498508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982a0c8-acb5-4ab3-a571-47641e83f4c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:58.500178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 120/241] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:26:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:58.504958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9243-0bbc-4e96-8608-ac732a8dd6ff None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.504958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9243-0bbc-4e96-8608-ac732a8dd6ff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.504958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad9243-0bbc-4e96-8608-ac732a8dd6ff None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.504958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 121/242] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:58.507930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad9243-0bbc-4e96-8608-ac732a8dd6ff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.526795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab5b5-e05d-48c6-8a59-3d3965a0727a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:58.573824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.575796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.576331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.576762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] PATH_INFO: `/v3/users/e7408bc380444c768e2ef4ed1a33a5cf` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.597061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] RBAC: Authorizing `identity:get_user(user_id=e7408bc380444c768e2ef4ed1a33a5cf)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:58.599088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe88f7-a9e7-4e12-8052-c7138b932fda None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:58.602274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 122/243] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:58 2022] GET /identity/v3/users/e7408bc380444c768e2ef4ed1a33a5cf => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:26:58.604975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1598958-2f70-4f42-bb4f-6032d5ec5271 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.605423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1598958-2f70-4f42-bb4f-6032d5ec5271 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.605848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1598958-2f70-4f42-bb4f-6032d5ec5271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.628261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a1598958-2f70-4f42-bb4f-6032d5ec5271 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:58.629989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a1598958-2f70-4f42-bb4f-6032d5ec5271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:58.642739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad9243-0bbc-4e96-8608-ac732a8dd6ff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:58.680800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.682377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.682911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.683417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.684357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:58.686130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:58.694077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:58.718393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 122/244] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:26:58 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:58.719911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.732554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 123/245] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:58.754622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e84e1-29af-4700-a210-bedabc23ae64 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.754813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e84e1-29af-4700-a210-bedabc23ae64 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.755016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e84e1-29af-4700-a210-bedabc23ae64 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.756803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 124/246] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:58.759454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:58.763440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.763741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.764197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.791346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:58.792623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:58.844428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76233585-41e8-4585-887e-fa85d83e7407 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:58.873974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 125/247] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:58.874889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:58.885856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:58.886933 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:58.887558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:58.888075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:58.966509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:58.968267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.001744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710069dc-0db5-48b9-8ad4-48f5030004ef None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.006108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.018304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.061121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.062124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.062432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.062735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.076770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:59.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:59.086140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:59.086953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 126/248] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:59.087630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.126632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 123/249] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:26:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:26:59.127483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.245578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21903cc-24ed-4fe9-be95-bd7d3887c93e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.260874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c113-2383-4359-b630-4ad241a8d7a8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.294661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.302448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.303211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.303511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.303791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.304559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:59.304873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.306275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.315783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.316122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 124/250] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.316928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.317081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.317372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.317491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.337688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:26:59.354366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:26:59.355313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 127/251] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:26:59.356929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.416100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.436385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a495b0-dbef-4ec7-a4d7-cb877e2a4930 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.473510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.474393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.474682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.474961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.476233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.477698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.479570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 125/252] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.480336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.559985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.587922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1762eb-a820-4c8f-be34-2679e8509378 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.612651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246b15-b600-4588-9cb1-b53c148fe0bc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.622360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.623925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.624328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.624703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] PATH_INFO: `/v3/users/257fdb9d95884e5ea16dcf1e36a380d7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.626170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] RBAC: Authorizing `identity:get_user(user_id=257fdb9d95884e5ea16dcf1e36a380d7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.627877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.629903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 128/253] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/users/257fdb9d95884e5ea16dcf1e36a380d7 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.630867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.655431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.656279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.656656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.657045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.657905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:59.658311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.659817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e3491-b481-4538-a6cd-55188abe37da None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.670554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 126/254] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.674995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738deaf0-4895-474f-89cd-9cc964af18c8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.674995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738deaf0-4895-474f-89cd-9cc964af18c8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.674995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738deaf0-4895-474f-89cd-9cc964af18c8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.674995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 127/255] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:26:59.675965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738deaf0-4895-474f-89cd-9cc964af18c8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.767419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.773353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42d94-8c30-46db-a6bb-7aa3f7a1b495 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.785112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738deaf0-4895-474f-89cd-9cc964af18c8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.820601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.821411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:59.821769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.822143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.822531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.822724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.823165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.823680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.824139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.824837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=e7408bc380444c768e2ef4ed1a33a5cf) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:59.825415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:59.826157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.827202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5343-6b07-46d1-8438-ae0e76c311a9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:59.828285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 128/256] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.833926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 129/257] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/role_assignments?user.id=e7408bc380444c768e2ef4ed1a33a5cf&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:59.834082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:59.836052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:59.836505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:59.836954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:59.839525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:59.863395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:59.865395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:59.962788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f0d75-b07c-4486-92b0-5e58f32b0a80 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:59.984229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 130/258] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:26:59.985244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.006355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:00.007700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:00.008150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:00.008625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:00.010049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:00.011745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:00.014310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 129/259] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:26:59 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:00.016222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.087895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:00.139907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73b017-f465-48dd-85f6-f55c4d149fd9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:00.186961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:00.187821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:00.188102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:00.188377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:00.206910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:00.216052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:00.217279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:00.218125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:00.225091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 130/260] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:00.226024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:00.227176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.242316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d8712-ced2-41dc-a9f7-37e2fd0d16ba None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:00.328374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:00.329645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:00.329929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:00.330207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/257fdb9d95884e5ea16dcf1e36a380d7/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:00.336412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, user_id=257fdb9d95884e5ea16dcf1e36a380d7, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:00.337885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:00.338652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed914e-5188-4a0a-acb4-673b0036ceba None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:00.357376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:00.376860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 131/261] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:26:59 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/257fdb9d95884e5ea16dcf1e36a380d7/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:00.377833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.393879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:00.394558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:00.394846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:00.395198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:00.546635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:00.550184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:00.636377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f78c-cf4a-421d-bf34-d10aff923578 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:00.721378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:00.722067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:00.722352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:00.722629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:00.723412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:00.723720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:00.725673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:00.747284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 132/262] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:00.752735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.780276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:00.841207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 131/263] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:00.856532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:00.887431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:00.958437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7af803-e3e5-4c25-843f-099a394910ab None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.010057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.010943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.011259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.011541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] PATH_INFO: `/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.012806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] RBAC: Authorizing `identity:get_user(user_id=ae80f1b2c9324bdeb1e3eb4eec88300c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.014329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.016328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 133/264] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:00 2022] GET /identity/v3/users/ae80f1b2c9324bdeb1e3eb4eec88300c => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:01.021773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.046542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ffaed-166b-4d9d-84e4-152424914799 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.125429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.126518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.126808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.127135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.128429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.132075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.134868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 132/265] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:00 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:01.141537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.183263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:01.217720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64063d4-ec39-4e44-8a79-9f0f1cc6687d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.285754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.286415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.286698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.286972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.305332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:01.330430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3825f-8e0c-43bf-be01-111447649e20 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.398085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.399670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.402211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.403245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.403614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.403899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.404755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=ae80f1b2c9324bdeb1e3eb4eec88300c) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:01.405068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.406553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166f0bc-3b00-41a5-b483-e31e61d67f82 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.418658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 133/266] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/role_assignments?user.id=ae80f1b2c9324bdeb1e3eb4eec88300c&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:01.482306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d80b-14ea-48d2-9c51-92c8f68af46b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.483013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d80b-14ea-48d2-9c51-92c8f68af46b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.483519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977d80b-14ea-48d2-9c51-92c8f68af46b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.485348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 134/267] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:01.492259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:01.493547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.493837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.494116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.518768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 134/268] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:01.527980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.538415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:01.539676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:01.657421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad76663-40c1-4103-82d7-6acfcd316621 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.679541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 135/269] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:01.710246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.711178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.711465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.711741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.713140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.715226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.718593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.728875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 135/270] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:01.734988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.792686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:01.822039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:01.845631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a8ea2-d87f-438f-a6df-b7233da65798 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.851388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a7ed7-ea0b-46b4-8c14-c0f6f75b1760 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:01.888327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.889423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:01.889643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.890057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.890368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.890630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.890828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.891259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.892204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=ae80f1b2c9324bdeb1e3eb4eec88300c) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:01.892635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:01.894415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cd550-c259-4250-9839-abe8986f792e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:01.901748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 136/271] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/role_assignments?user.id=ae80f1b2c9324bdeb1e3eb4eec88300c&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=4169efc51b3041918162437c5960c781 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:01.903911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcccef-63ae-4a42-a1ce-a36b7c52af92 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.903911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcccef-63ae-4a42-a1ce-a36b7c52af92 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.903911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcccef-63ae-4a42-a1ce-a36b7c52af92 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.904811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:01.905708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 137/272] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:01.911529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:01.912025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:01.912435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:01.915245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:01.916209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:01.917116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:01.920140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 136/273] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:01.925733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:01.931727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:01.933251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:02.010338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 138/274] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:02.083448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d1ffe-cd20-4c73-b330-12c4a1dd3595 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:02.097091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:02.168948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:02.169578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.169860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.170131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.170907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:02.171249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:02.172681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:02.175362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:02.190723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 137/275] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:02.196263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:02.201676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:02.237868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c138c4-802a-40e6-bff0-de9a08673982 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:02.275297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:02.276216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.276505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.276780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.278010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:02.279638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:02.281723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 139/276] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:02.287234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:02.329054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bec71-ce43-47d5-a992-7dfbf69895fd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:02.379141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:02.380040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.380317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.380586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] PATH_INFO: `/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.400177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb8caa54f3f41cfa8287968fbaadbb9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:02.403810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:02.406117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 138/277] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:02 2022] GET /identity/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:02.411503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:02.417867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec0d2-1b81-4735-bd1e-e6e504bf9f04 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:02.437796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:02.474772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:02.476468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.477087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.477721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.479148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:02.481538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf5fc3-ffb8-445e-9b78-e66458b40953 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:02.509177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:02.514374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 140/278] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:02 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:02.559193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb99efa-4d39-4c8c-9cef-fccf024ab396 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:02.598680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:02.599398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.599734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.600042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.600903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, user.id=ecb8caa54f3f41cfa8287968fbaadbb9, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:02.601219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:02.602751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035c47-9be0-4bf2-831d-8fb966d3e956 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:02.609330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 139/279] 10.222.0.74 () {56 vars in 1308 bytes} [Thu Jun 16 16:27:02 2022] GET /identity/v3/role_assignments?user.id=ecb8caa54f3f41cfa8287968fbaadbb9&scope.system=all&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:02.719876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7cfe-a051-402d-8d42-35fa84004dae None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.720062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7cfe-a051-402d-8d42-35fa84004dae None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.720369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7cfe-a051-402d-8d42-35fa84004dae None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.721947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 141/280] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:02.728012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:02.728638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:02.729112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:02.747977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:02.749603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:02.828423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 140/281] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:02.851815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7cfe-a051-402d-8d42-35fa84004dae None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:02.972165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7cfe-a051-402d-8d42-35fa84004dae None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.017559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.018366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.018646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.018916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.031334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:03.039411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:03.040305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:03.041550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 142/282] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:03.049308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.121810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:03.184159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a27c6-f273-4031-9391-6706f9c693c1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.233815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.234412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.234695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.234968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.235763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:03.236067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:03.237486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:03.251286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 141/283] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:03.257204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.292874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:03.381837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91cc77-4e68-4fe1-a51c-b3b7089745d0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.417261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.418144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.418424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.418698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] PATH_INFO: `/v3/users/e7408bc380444c768e2ef4ed1a33a5cf` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.419945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] RBAC: Authorizing `identity:get_user(user_id=e7408bc380444c768e2ef4ed1a33a5cf)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:03.421420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f8f98-0bf7-4d94-8556-2b890bf3db9b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:03.423388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 143/284] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:03 2022] GET /identity/v3/users/e7408bc380444c768e2ef4ed1a33a5cf => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:03.432225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.547133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e5e0-7a91-42fc-8209-bdf9d676097a None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.547304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e5e0-7a91-42fc-8209-bdf9d676097a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.547536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e5e0-7a91-42fc-8209-bdf9d676097a None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.549157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 144/285] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:03.555540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.556102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.556581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.573736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6742ca5-42d8-4d62-80c6-fa49d6a354ca None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.574718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:03.576346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:03.596725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:03.632428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.633316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.633600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.633877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.644979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:03.644979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:03.644979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 142/286] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:03 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:03.649462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.672866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 145/287] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:03.714274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.747465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:03.793115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f7883-ba8c-4bca-bb6f-a5bb73871a04 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.827553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.828370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.828690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.828915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/e7408bc380444c768e2ef4ed1a33a5cf/roles/c1d2e0a6311d4b40ad03045bc9d276dc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.830881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, user_id=e7408bc380444c768e2ef4ed1a33a5cf, role_id=c1d2e0a6311d4b40ad03045bc9d276dc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:03.832377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:03.838645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf55b4-712a-4228-b2ab-7d4da28d8ac7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:03.857001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 143/288] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:03 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/users/e7408bc380444c768e2ef4ed1a33a5cf/roles/c1d2e0a6311d4b40ad03045bc9d276dc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:27:03.878257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:03.879947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:03.880496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:03.880965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:03.893938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:03.902776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:03.903703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:03.904482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:03.907442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 146/289] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:03.915114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:03.993692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:04.074372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4fd23-b352-4bcf-af08-742aa7bb6984 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:04.130489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:04.131195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:04.131620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:04.132206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:04.132657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:04.132967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:04.134455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:04.147511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 144/290] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:04.153191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:04.274395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:04.291191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0a339-304f-45dd-a3cd-3a8cf15d15b5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:04.328326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:04.329230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:04.329515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:04.329794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] PATH_INFO: `/v3/users/257fdb9d95884e5ea16dcf1e36a380d7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:04.331012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] RBAC: Authorizing `identity:get_user(user_id=257fdb9d95884e5ea16dcf1e36a380d7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:04.332562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:04.335279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 147/291] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:04 2022] GET /identity/v3/users/257fdb9d95884e5ea16dcf1e36a380d7 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:04.340041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:04.477888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259b6b-8c33-4725-800c-50d2f620099b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:04.485494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:04.525110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:04.525778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:04.526059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:04.526335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:04.630550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:04.632225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:04.764053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 145/292] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:04.769396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:04.920994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a7e85-940f-4916-9b5a-31f05a3a39e2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:04.966802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:04.967733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:04.968023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:04.968303 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:04.969645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:04.971382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4e8ed-c43e-4e41-a7c7-b64553c2b42a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:04.974038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 148/293] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:04 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:04.980264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:04.995450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.051221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7856f4-2474-47a8-b335-28d8e0a6cecf None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.051521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7856f4-2474-47a8-b335-28d8e0a6cecf None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.051714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7856f4-2474-47a8-b335-28d8e0a6cecf None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.055156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 149/294] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:05.059380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac7e2-2aa0-4a5a-80d2-c97bab868243 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.059728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac7e2-2aa0-4a5a-80d2-c97bab868243 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.059932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac7e2-2aa0-4a5a-80d2-c97bab868243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.100333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1b7ac7e2-2aa0-4a5a-80d2-c97bab868243 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:05.101661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1b7ac7e2-2aa0-4a5a-80d2-c97bab868243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:05.108928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453dfdc-37d2-4e8c-8d18-77a181b933c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.134297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.149140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:05.150266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.150852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.151383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.152444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=257fdb9d95884e5ea16dcf1e36a380d7) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:05.153086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:05.154857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:05.161829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 146/295] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:04 2022] GET /identity/v3/role_assignments?user.id=257fdb9d95884e5ea16dcf1e36a380d7&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:05.239922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 150/296] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:05.278620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.363761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.371141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57926b03-b1d6-4a10-bc79-78ec763b9264 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.371501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57926b03-b1d6-4a10-bc79-78ec763b9264 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.371724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57926b03-b1d6-4a10-bc79-78ec763b9264 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.374682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 151/297] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:05.379126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.379422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.379711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.396904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05644e8-1282-4614-b2a9-e144dd22b3a5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.415625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:05.416853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:05.450003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:05.451262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.451761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.452227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.478091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:05.487607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:05.488476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:05.489247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:05.492076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 147/298] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:05.497015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 152/299] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:05.498877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.534714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.624937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0ce33-816e-4e2d-b00a-32367ed764b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.649064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.662257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.670532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:05.671251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.671538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.671815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.672572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:05.672883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:05.673051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed764-7d6a-4c97-85cf-8394cc7bfbc7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.674276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc345-2b49-45d4-927c-85f02d09a970 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:05.687918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 148/300] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:05.690346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6229fbf-c607-4d04-abee-af2487311656 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.690946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6229fbf-c607-4d04-abee-af2487311656 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.691465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6229fbf-c607-4d04-abee-af2487311656 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.693419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 149/301] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:05.694227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6229fbf-c607-4d04-abee-af2487311656 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.714318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:05.715160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.715454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.715734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.727335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b2115067-98ea-4841-aa1d-d369ff2c1741 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:05.735033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:05.735894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:05.736673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 153/302] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:05.737618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.737736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.738004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.759178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:05.760373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:05.816429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6229fbf-c607-4d04-abee-af2487311656 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.833689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 154/303] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:05.840518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.855421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:05.856320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:05.856602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:05.858156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] PATH_INFO: `/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:05.858368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] RBAC: Authorizing `identity:get_user(user_id=ecb8caa54f3f41cfa8287968fbaadbb9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:05.859820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:05.861620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 150/304] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:05.863080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:05.924962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.926709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:05.962438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb6c6a-c9e9-4aff-ab82-ed1604e7727d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:05.970145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d12c7-f7e4-42e4-bfe4-adfd4ce2330f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:06.003145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.003816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.004094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.004355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.005100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:06.005404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.005636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.006514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.006883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.007028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.007263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.008453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.009938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.011798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 151/305] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:06.020392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 155/306] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:06.020536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:06.020613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:06.125898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aeebf9-e19c-4dfe-b4cf-72e8d78357db None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:06.133211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430afbb-4ebf-461c-8c80-a3840772da5e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:06.161018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.161697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.161985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.162272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] PATH_INFO: `/v3/groups` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.163006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.164473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.165344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:06.167705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:06.175435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.176288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.176631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.176864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] PATH_INFO: `/v3/system/users/ecb8caa54f3f41cfa8287968fbaadbb9/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.178482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ecb8caa54f3f41cfa8287968fbaadbb9, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.179914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.182204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 156/307] 10.222.0.74 () {60 vars in 1134 bytes} [Thu Jun 16 16:27:06 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:06.182937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:06.201230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 152/308] 10.222.0.74 () {58 vars in 1334 bytes} [Thu Jun 16 16:27:06 2022] PUT /identity/v3/system/users/ecb8caa54f3f41cfa8287968fbaadbb9/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:06.312617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0408d-a8c2-4aad-a6ac-bd3f986ecb6f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:06.355733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.356662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.356951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.357231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] PATH_INFO: `/v3/users/e5bed34630db42b880d8efb71af02209` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.375346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] RBAC: Authorizing `identity:get_user(user_id=e5bed34630db42b880d8efb71af02209)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.376908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f5e53-8b08-4841-b012-40b86a8e172d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.383200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 157/309] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:06 2022] GET /identity/v3/users/e5bed34630db42b880d8efb71af02209 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:06.385805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:06.506322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eec08f-1989-42b6-bd56-bd2388ad6001 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:06.545634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:06.546307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.546594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.546867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.547883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, user.id=e5bed34630db42b880d8efb71af02209, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:06.548155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:06.549625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3cd2d-0459-482d-ba49-c7759ef46f26 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:06.556091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 153/310] 10.222.0.74 () {56 vars in 1308 bytes} [Thu Jun 16 16:27:06 2022] GET /identity/v3/role_assignments?user.id=e5bed34630db42b880d8efb71af02209&scope.system=all&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:06.765070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1a91-4cf9-45ab-bd38-2634348b6af5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.765421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1a91-4cf9-45ab-bd38-2634348b6af5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.765597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1a91-4cf9-45ab-bd38-2634348b6af5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.767153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 158/311] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:06.772659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:06.772935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:06.773207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:06.790692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:06.791904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:06.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:06.870781 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 154/312] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:06.893389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b1a91-4cf9-45ab-bd38-2634348b6af5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:06.957545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:07.010665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b1a91-4cf9-45ab-bd38-2634348b6af5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:07.047427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:07.048198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:07.048555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:07.048838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:07.062433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:07.070702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:07.071598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:07.072366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 159/313] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:07.077951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:07.187128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8504e4-6d60-4e6e-b800-dd7ec1accb41 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:07.225326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:07.226301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:07.226590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:07.226867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:07.227707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:07.228016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:07.229420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:07.243174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 155/314] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:07.244966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:07.348411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad2475-aa9c-4298-b9af-6d62b05b0c37 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:07.363423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:07.388880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:07.389830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:07.390109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:07.390378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] PATH_INFO: `/v3/users/e7408bc380444c768e2ef4ed1a33a5cf` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:07.391668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] RBAC: Authorizing `identity:get_user(user_id=e7408bc380444c768e2ef4ed1a33a5cf)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:07.393181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:07.395129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 160/315] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:07 2022] GET /identity/v3/users/e7408bc380444c768e2ef4ed1a33a5cf => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:07.400768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:07.456180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:07.514208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631575c-66a5-4d88-a5a4-ef47e5c30405 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:07.554757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:07.555755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:07.556269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:07.556732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:07.629316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:07.630738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:07.680395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:07.739235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 156/316] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:07.745278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:07.858245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056901c4-764f-4958-ac2f-77a05c26a96b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:07.893348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:07.894132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:07.894421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:07.894694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:07.895926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:07.897805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5db1-322a-471a-9e83-9fd1f14391d5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:07.899905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 161/317] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:07 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:07.905714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:08.009989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3dde9-c9ed-4eb0-8881-112493a2e669 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:08.045797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:08.046826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.047354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.047865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.048915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab, user.id=e7408bc380444c768e2ef4ed1a33a5cf) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:08.049444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:08.051123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b741-75ce-408e-a0bb-d894fbcd298b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:08.058392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 157/318] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:07 2022] GET /identity/v3/role_assignments?user.id=e7408bc380444c768e2ef4ed1a33a5cf&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:08.520434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a40679-1366-4f7e-af5b-7e0fd6bbf89e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.520975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a40679-1366-4f7e-af5b-7e0fd6bbf89e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.520975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a40679-1366-4f7e-af5b-7e0fd6bbf89e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.522607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 162/319] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:08.527752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.528193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.528265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.540284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68c86-ad9f-486b-9a3c-35e8c84945be None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.540605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68c86-ad9f-486b-9a3c-35e8c84945be None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.540840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68c86-ad9f-486b-9a3c-35e8c84945be None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.542449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 163/320] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:08.549161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.549421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.549823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.552379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:08.553664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:08.563399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:08.566911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:08.568128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:08.613988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:08.631732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 158/321] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:08.639694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 164/322] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:08.654802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:08.662575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:08.761502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197666c-3d1c-40ab-9da4-48605f4bed89 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:08.777206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339ec4f-e975-4298-8044-b4ca68fc6ca0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:08.801785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:08.802601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.802886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.803231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.806593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:08.808768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:08.811150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 159/323] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:08.816540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:08.823376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:08.824193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.824478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.824754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.828962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:08.837195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ad792816-df19-4fd2-8bc5-48d179fd9798 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:08.845046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:08.846052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:08.846979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 165/324] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:08.848012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ddec-2ba7-42a8-a6ba-30a71baf8c91 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.848012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ddec-2ba7-42a8-a6ba-30a71baf8c91 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.848138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514ddec-2ba7-42a8-a6ba-30a71baf8c91 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.853368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 166/325] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:08.853368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514ddec-2ba7-42a8-a6ba-30a71baf8c91 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:08.893045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:08.920565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbe5fa-4e34-4935-9627-f69b47cce930 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:08.957142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:08.957349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514ddec-2ba7-42a8-a6ba-30a71baf8c91 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:08.957817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.958095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.958408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] PATH_INFO: `/v3/groups` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.959183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:08.960692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44e8d3-e7b2-41f5-9692-ae22b83ac558 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:08.977875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 160/326] 10.222.0.74 () {60 vars in 1134 bytes} [Thu Jun 16 16:27:08 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:27:08.979173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:08.979465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:08.979746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:08.998316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:08.999605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:09.013141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.013949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.014242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.014522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.015414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:09.015777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.017471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.028582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 167/327] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:09.033971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.078952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:09.083752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 161/328] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:09.106900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.152661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:09.157012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafcfc4-a7fe-494c-bab4-03bbf29c3900 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.197132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.198102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.198388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.198665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] PATH_INFO: `/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.200027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] RBAC: Authorizing `identity:get_user(user_id=ecb8caa54f3f41cfa8287968fbaadbb9)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.201425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.203474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 168/329] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:09 2022] GET /identity/v3/users/ecb8caa54f3f41cfa8287968fbaadbb9 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:09.208980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.233793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fb2a0-4dda-4a23-96b9-8a88dce2a0a6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.277340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.279525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.279525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.279525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.291657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:09.299504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:09.300434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:09.301191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 162/330] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:09.307643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.337483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f099c-c1e3-485e-97e1-7ce1b5c9d8ac None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.378427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.379822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.379822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.379822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.380641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] RBAC: Adding query filter params (role.id=a0c9d2da81de41eb955b1392d9a2e766, user.id=ecb8caa54f3f41cfa8287968fbaadbb9, scope.system=all) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:09.383493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.386867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.388403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:09.401911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 169/331] 10.222.0.74 () {56 vars in 1308 bytes} [Thu Jun 16 16:27:09 2022] GET /identity/v3/role_assignments?user.id=ecb8caa54f3f41cfa8287968fbaadbb9&scope.system=all&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:09.432843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275faff3-e25e-4e58-909a-dd6fd68a89dd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.471977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.472669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.472953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.473229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.473989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:09.474296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.475806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.485785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 163/332] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:09.491617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.530542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:09.617174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4d506-b89c-4c52-9b1c-160d02a245ab None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.653857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.654751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.655037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.655345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] PATH_INFO: `/v3/users/e5bed34630db42b880d8efb71af02209` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.656573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] RBAC: Authorizing `identity:get_user(user_id=e5bed34630db42b880d8efb71af02209)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.658038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e7efc-42ac-4c0c-b32b-51196d912671 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.659733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 170/333] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:09 2022] GET /identity/v3/users/e5bed34630db42b880d8efb71af02209 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:09.665907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:09.773394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb2e5-3067-4096-aa3e-8ff3f76ee521 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:09.798421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:09.810447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:09.811389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:09.811657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:09.811939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] PATH_INFO: `/v3/system/users/e5bed34630db42b880d8efb71af02209/roles/c1d2e0a6311d4b40ad03045bc9d276dc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:09.813488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e5bed34630db42b880d8efb71af02209, role_id=c1d2e0a6311d4b40ad03045bc9d276dc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:09.814887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8449f35-a445-4bb9-b89f-1afb96fe4508 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:09.825037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 164/334] 10.222.0.74 () {58 vars in 1334 bytes} [Thu Jun 16 16:27:09 2022] PUT /identity/v3/system/users/e5bed34630db42b880d8efb71af02209/roles/c1d2e0a6311d4b40ad03045bc9d276dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:11.579817 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bdc06-0986-426e-b2ea-a89bd52b9ce9 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:11.580325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bdc06-0986-426e-b2ea-a89bd52b9ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:11.580399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bdc06-0986-426e-b2ea-a89bd52b9ce9 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:11.582044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 171/335] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:11.587277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:11.588155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:11.588635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:11.612520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:11.614080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:11.697722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 165/336] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:11.734779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bdc06-0986-426e-b2ea-a89bd52b9ce9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:11.857061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:11.868677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bdc06-0986-426e-b2ea-a89bd52b9ce9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:11.906584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:11.907451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:11.907738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:11.908014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:11.919297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:11.927076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:11.928423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:11.929195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:11.930583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 172/337] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:11.936245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:12.069964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516738b-b866-45a6-9a99-a2737fc5a131 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:12.089890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:12.119882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:12.120505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.120789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.121074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.121844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:12.122155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:12.123599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dc99f-a821-424f-a12b-ef503062de5e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:12.138535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 166/338] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:12.139969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:12.257462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d89a7e-a0c1-45a9-9ec2-4cfaacfbbac2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.258113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d89a7e-a0c1-45a9-9ec2-4cfaacfbbac2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.258619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d89a7e-a0c1-45a9-9ec2-4cfaacfbbac2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.260635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 167/339] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:12.266005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.266561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.267069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.295255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e650f9d-9792-41db-aebf-4447c0a8e156 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:12.308117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:12.309726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:12.341053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:12.341966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.342436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.342914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.383516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:12.456918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 168/340] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:12.474511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:12.476165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:12.512057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:12.689502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778d6d9-d2f2-4974-828b-c5fd2513e26d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:12.702820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 173/341] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:12.712083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:12.731447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:12.742442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:12.743295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.743580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.743856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.760622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-be5d1a97-4494-4f02-a66b-e7a609f80d55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:12.768652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:12.769542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:12.770748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 169/342] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:12.772277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a07f17-4fce-4360-8e33-7b19fa842474 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.772978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a07f17-4fce-4360-8e33-7b19fa842474 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.772978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a07f17-4fce-4360-8e33-7b19fa842474 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.777507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 170/343] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:12.781740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a07f17-4fce-4360-8e33-7b19fa842474 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:12.825506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:12.905155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8202-3b34-49af-a7db-5f6c1d931332 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:12.971897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789561b-deda-4734-8211-64104c13246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:12.973101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789561b-deda-4734-8211-64104c13246c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.974231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789561b-deda-4734-8211-64104c13246c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.974705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789561b-deda-4734-8211-64104c13246c None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.976377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789561b-deda-4734-8211-64104c13246c None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:12.979349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789561b-deda-4734-8211-64104c13246c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:12.983088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 174/344] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:12 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:12.984665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:12.987040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:12.988109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:12.993639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a07f17-4fce-4360-8e33-7b19fa842474 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.019067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:13.022124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:13.063741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.064794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.065298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.065768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.066721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:13.068396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.070194 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.088286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 171/345] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:13.090416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.141735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 175/346] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:13.143636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.159420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.245445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734b016-cd09-4a57-ac03-2907b26b418f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.247508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.292589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.293774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.294304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.297321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.304847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.306521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.306854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60745f-7f32-4d7c-8d59-8ccedd519fe1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.308419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 172/347] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:13.310312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.354908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.355851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.356272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.356679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.436051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ada9b-5527-422a-a076-fdef1ea21f76 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.438484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.440241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74d5a-57dc-4bde-a332-a226963948c7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.447307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.478480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.479579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.480105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.480573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.492819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:13.501389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:13.504490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:13.505545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:13.506428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:13.508864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 173/348] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:13.510034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.554714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 176/349] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:13.556324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204690-5376-4090-8431-e4c879c8d80c None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.556753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204690-5376-4090-8431-e4c879c8d80c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.557165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204690-5376-4090-8431-e4c879c8d80c None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.558591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 177/350] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:13.559790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204690-5376-4090-8431-e4c879c8d80c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.665567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d23dc-f222-4bde-b839-e50a6526f8fc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.710730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204690-5376-4090-8431-e4c879c8d80c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.714602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.715905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.716341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.716754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.717730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:13.718186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.719813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c9a1-0c64-4f28-9d40-bf4c195c0ca0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.726761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 174/351] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:13.728316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b39f-90cd-456c-b2a4-cc56c18ce5a2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.728743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b39f-90cd-456c-b2a4-cc56c18ce5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.729158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64b39f-90cd-456c-b2a4-cc56c18ce5a2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.730718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 175/352] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:13.731871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b39f-90cd-456c-b2a4-cc56c18ce5a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:13.750078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.751005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.751556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.752029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.752973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:13.753470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.755077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73832181-b78b-4c9b-b4ad-758c1e24fa7e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.755553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.767780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 178/353] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:13.769678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b233-8055-40ef-ae85-aef2133612cd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.770173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b233-8055-40ef-ae85-aef2133612cd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.770647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100b233-8055-40ef-ae85-aef2133612cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.797729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6100b233-8055-40ef-ae85-aef2133612cd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:13.799540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6100b233-8055-40ef-ae85-aef2133612cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:13.811697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:13.892967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64b39f-90cd-456c-b2a4-cc56c18ce5a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:13.903757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 179/354] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:13.905563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.906040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.906496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.928166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:13.931396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:13.936463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:13.937523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:13.937904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:13.938275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:13.939721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:13.942122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:13.945081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 176/355] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:13.946133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.031610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 180/356] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:14.032858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.066515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.084813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2578519-f905-49c0-95f7-df767d884c78 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.138797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.139295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.139540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.139651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.140251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.173183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61019c3b-f7eb-4864-945b-26f16e574775 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.220109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.221054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.221335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.221607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.229489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.231484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6eb5c-08ef-46ae-b3bf-b735e62d090e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:14.238151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:14.250893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:14.251887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 16:27:14.252646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:14.253219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 181/357] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:14.253779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.350398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.375093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 177/358] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:14.376387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058b-1aa0-4ab5-b4e9-7a4cbe4e3b7b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.376680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058b-1aa0-4ab5-b4e9-7a4cbe4e3b7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.376943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f058b-1aa0-4ab5-b4e9-7a4cbe4e3b7b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.378350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 178/359] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:14.379172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f058b-1aa0-4ab5-b4e9-7a4cbe4e3b7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.396865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f91f8-d75d-46ea-975f-7fa744baa6ef None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.441391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.442660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.443226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.443793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.446293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.447972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:14.449783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 182/360] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:14.450715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.453655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.510570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f058b-1aa0-4ab5-b4e9-7a4cbe4e3b7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.563443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.564382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.564663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.565746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.581121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:14.590652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:14.591669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:14.592440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 179/361] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:14.593082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.602634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1508a9-eeae-4970-bb2f-22d3226c65f9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.643731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.644414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.644696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.644971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.645736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:14.646045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.647667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0f28a-181d-452a-9067-871c3235f2e9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:14.653420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.656887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 183/362] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:14.658224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.658514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.658790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.677075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:14.678422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:14.714543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c2a1-1624-4d42-ad58-10ab5eab83b1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.741996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.754201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 184/363] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:14.756065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.761838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.763271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.763701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.764182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.765629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.767428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:14.774215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 180/364] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:14.775431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.886437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd08ce-5bed-48ff-9878-a487a31fc825 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.932801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.933493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.933782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.934066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.934838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7be3eb-7936-43f3-bd75-e104d8da9621 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:14.934997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=4169efc51b3041918162437c5960c781, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:14.935286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.937022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:14.944139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 185/365] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=4169efc51b3041918162437c5960c781 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:14.947147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:14.956426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:14.986430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:14.987336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:14.987725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:14.988107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:14.989302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:14.989754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:14.992222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.004512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 181/366] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.005390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.034723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.094020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0e915-de8b-4549-a014-7dea2aeb9a41 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.143805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.144711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.145400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] PATH_INFO: `/v3/users/e5bed34630db42b880d8efb71af02209` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.146761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] RBAC: Authorizing `identity:get_user(user_id=e5bed34630db42b880d8efb71af02209)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.148287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.150262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 186/367] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:27:14 2022] GET /identity/v3/users/e5bed34630db42b880d8efb71af02209 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.151044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.160737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ea96-f818-46c6-bdb7-8e867d0c8c69 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.202294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.203202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.203486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.203763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.204947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.206414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.208069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 182/368] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.208831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.283578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.286731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e71e3f-89bd-4a20-8644-60db092624ac None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.316972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.327604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.328061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.328340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.328610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.344460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:15.352771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:15.353670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:15.354627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 187/369] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:15.361544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.361795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a54be-54a8-475e-8ba6-2cb0faac1383 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.407399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.408292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.409269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.409720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.481516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35163f6-ca5a-4bc2-ac38-49bacb26e309 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.502079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.503843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.525437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.526520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.527258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.527684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.528673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] RBAC: Adding query filter params (role.id=c1d2e0a6311d4b40ad03045bc9d276dc, user.id=e5bed34630db42b880d8efb71af02209, scope.system=all) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:15.529133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.531265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.546641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 188/370] 10.222.0.74 () {56 vars in 1308 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/role_assignments?user.id=e5bed34630db42b880d8efb71af02209&scope.system=all&role.id=c1d2e0a6311d4b40ad03045bc9d276dc => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.547830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.572260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.611456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.657062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 183/371] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:15.658590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.687229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd284d6-c611-4a1f-98b1-a82d8f3874d1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.730021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.731241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.731241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.731540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.732287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:15.732601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.734091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.742463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 189/372] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.747833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.795567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e83898-064f-46cf-a198-0c2c0b0c251f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.848652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:15.849355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:15.849633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:15.849935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:15.851890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:15.852448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:15.854069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:15.864591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 184/373] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:15.873440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:15.886332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:15.971087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874f25-efd7-48c8-adcc-64d5879853c6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:15.987703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.048943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.049873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.050154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.050433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.052319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.053882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.055658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 190/374] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:16.061254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.068607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa7e08-c2be-4644-a03d-668981d4cd5b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.120394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.121484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.121989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.122489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.219535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27e1d6-eccf-470d-8eda-9422d95d4607 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.220989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.222518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfbc07-157a-4422-9162-d66173924a83 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.230931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.258039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.259182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.259883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.261256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.262757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.264461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3a608-43ea-40a0-be95-64ea10cefdb4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.271432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 191/375] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:16.273901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a05a-974b-4afb-9956-e043277526ed None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.274182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a05a-974b-4afb-9956-e043277526ed None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.274457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0a05a-974b-4afb-9956-e043277526ed None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.276106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 192/376] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:16.276991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0a05a-974b-4afb-9956-e043277526ed None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.333748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.337033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 185/377] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:16.338505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.338931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.343617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.362931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:16.364538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:16.395610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0a05a-974b-4afb-9956-e043277526ed None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.434104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.434768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.435106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.435350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.436193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=4169efc51b3041918162437c5960c781, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:16.436510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.438023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.438608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 186/378] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:16.439798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.449085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 193/379] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=4169efc51b3041918162437c5960c781 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:16.470033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.481348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581e3d1-0067-42d8-9093-08b992867f7f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.633978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.634857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.635170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.635450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] PATH_INFO: `/v3/projects/f0df30bd71bf4670afbc831b314be545` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.638005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] RBAC: Authorizing `identity:get_project(project_id=f0df30bd71bf4670afbc831b314be545)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.639533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.641367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 187/380] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/projects/f0df30bd71bf4670afbc831b314be545 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:16.648849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.666408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57a599-5013-435b-989f-3bcf5ffb39bd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.683440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.740625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.741808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.742307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.742779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.755159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:16.765721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:16.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:16.772226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:16.772926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c86b-fa76-4d88-8efd-c27f6f639918 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:16.772926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:16.775104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 194/381] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:16.780723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.810084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:16.810967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:16.811605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:16.811887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] PATH_INFO: `/v3/groups/098d615f11ba4b6fba3cab29ae86078f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:16.823536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] RBAC: Authorizing `identity:get_group(group_id=098d615f11ba4b6fba3cab29ae86078f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:16.825027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:16.826936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 188/382] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/groups/098d615f11ba4b6fba3cab29ae86078f => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:16.833069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:16.972285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a86e16-3f44-41da-8a24-e7feaa2aa53c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.016337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.017004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.017305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.017588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.018355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:17.018671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.020232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.031309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 195/383] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:17.036786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:17.062350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3d69d-420a-4b6b-bcaa-4654841832eb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.078568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:17.149219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.149892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.150178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.150453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.151330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] RBAC: Adding query filter params (group.id=098d615f11ba4b6fba3cab29ae86078f, role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=f0df30bd71bf4670afbc831b314be545) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:17.151647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.153106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.166138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 189/384] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:16 2022] GET /identity/v3/role_assignments?group.id=098d615f11ba4b6fba3cab29ae86078f&scope.project.id=f0df30bd71bf4670afbc831b314be545&role.id=279eea356bbd45f39595ccaa1656385a => generated 260 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:17.170422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f698f-0cd5-445e-9a51-3a56271c46fc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.214673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.215935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.216399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.216878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.218336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.220156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.222596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 196/385] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:17 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:17.231113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:17.288532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:17.386607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400d6aa-1936-47cc-891e-d943142c0a3a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.432355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.433239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.433528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.433804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.435010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.436500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7acef-f813-4bfd-bb5b-d1dfe0a332d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.438565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 190/386] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:17 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:17.444449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:17.617586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:17.622117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd91c1-07e7-4801-9d34-caf71725efd1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.666548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.667482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.667773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.668055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/groups/b20ee172ddec429a800b171f92197835/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.672437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, group_id=b20ee172ddec429a800b171f92197835, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.673919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdd7ac-197b-4fba-a84e-041c4e9d3fe6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.692409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 197/387] 10.222.0.74 () {58 vars in 1442 bytes} [Thu Jun 16 16:27:17 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/groups/b20ee172ddec429a800b171f92197835/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:17.770469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ac2b-0d18-43f3-9917-12278fd74618 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.771163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ac2b-0d18-43f3-9917-12278fd74618 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.771679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831ac2b-0d18-43f3-9917-12278fd74618 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.773485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 191/388] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:17.779029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.779551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.779651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.803369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:17.805163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:17.897832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 198/389] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:17.921701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831ac2b-0d18-43f3-9917-12278fd74618 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.042013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:18.047618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831ac2b-0d18-43f3-9917-12278fd74618 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:18.088300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:18.089046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.089327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.089600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.105683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:18.113884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:18.114738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:18.116210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 192/390] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:18.121807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.265037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:18.282023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8343b90-8ef2-481f-a1f7-03bcf8da6c0c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:18.318291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:18.319440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.319931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.320434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.321403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:18.321910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:18.323536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:18.332981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 199/391] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:18.339806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.489301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593e1f-d726-437e-87c3-89779bc4d2b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:18.536160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:18.537048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.537390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.537670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.543847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:18.545358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3837-8cc5-4421-a619-d13081d6d8d5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:18.547419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 193/392] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:18 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:18.553629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.679338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585821f-6f90-4933-a892-70d8c935f33a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:18.690785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b399c2-36ae-4f81-b824-0b49cbdb676b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.690965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b399c2-36ae-4f81-b824-0b49cbdb676b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.691188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b399c2-36ae-4f81-b824-0b49cbdb676b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.692772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 194/393] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:18.700511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.700783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.701064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.717821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:18.719110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.719599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.720076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.721965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:18.723680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:18.726807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 200/394] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:18 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:18.735228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:18.736225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.736461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:18.762932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:18.764495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:18.824040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 195/395] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:18.847319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:18.920549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c05a3-add5-4861-b586-eda89449ecaf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:18.987862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:18.988760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:18.989052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:18.989329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/groups/b20ee172ddec429a800b171f92197835/roles/4169efc51b3041918162437c5960c781` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:18.993809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed718629b84987b6d5cb7cfbc28c3d, group_id=b20ee172ddec429a800b171f92197835, role_id=4169efc51b3041918162437c5960c781)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:18.995625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:19.022217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 201/396] 10.222.0.74 () {58 vars in 1442 bytes} [Thu Jun 16 16:27:18 2022] PUT /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d/groups/b20ee172ddec429a800b171f92197835/roles/4169efc51b3041918162437c5960c781 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:19.026034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8d630-201a-4b02-982f-1ebbaf2280f2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.064105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.064939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.065353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.065682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.077379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b2214c11-bd60-435d-9e9b-4ec160ccf3ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:19.086603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:19.087576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:19.088501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 196/397] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:19.096365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.241147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:19.271725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321872-fe37-40cc-8611-4dda38fad0db None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.303826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb752-7993-445d-89fe-c6324e9279ce None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.304465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb752-7993-445d-89fe-c6324e9279ce None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.304953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecb752-7993-445d-89fe-c6324e9279ce None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.306766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 197/398] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:19.312919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.313550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.314031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.318537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.319369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.319730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.320020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.320792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:19.321106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:19.322502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:19.341822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 202/399] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:19.346678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:19.348439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.348551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:19.363943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:19.463591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbd58e-cb48-4e12-9254-527cf5c33e1a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.465344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 198/400] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:19.504543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.504729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.505329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.505611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.505888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.507197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:19.508663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:19.510621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 203/401] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:19 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:19.518105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.535406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:19.689312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2a7d8-2df4-4f72-a449-7937a9792ae0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.705945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b4b22-07eb-47d6-8298-887b2ed72e26 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.741278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.742534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.743318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.743886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.745676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:19.747401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:19.749678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 204/402] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:19 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:19.756207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.776446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.777379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.777670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.777952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.803202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:19.811997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:19.812881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:19.813629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:19.819622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 199/403] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:19.826104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:19.845533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:19.848530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:19.871015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb72e3-d85c-4125-922f-459bd4f5911c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:19.908344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:19.909629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:19.910110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:19.910595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/groups/b20ee172ddec429a800b171f92197835/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:19.912903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, group_id=b20ee172ddec429a800b171f92197835, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:19.914618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c452d3-cf4e-415a-be63-776bc7e0fefe None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:19.937764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 205/404] 10.222.0.74 () {58 vars in 1442 bytes} [Thu Jun 16 16:27:19 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/groups/b20ee172ddec429a800b171f92197835/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:27:20.008971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b78b-57ba-492f-bc4e-12b821ab9808 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.029496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aed7a-f16b-45d8-8174-b363e12d3e0f None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.030129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aed7a-f16b-45d8-8174-b363e12d3e0f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.030604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181aed7a-f16b-45d8-8174-b363e12d3e0f None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.032429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 206/405] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:20.050459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.051165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.051644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.080501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:20.083480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:20.092512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.093911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.094392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.094861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.096485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:20.097033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.098705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.110557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 200/406] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:20.119091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.196399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 207/407] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:20.216101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:20.219213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.252473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1843e7-a8f0-4d3a-a147-bebd36774796 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.289483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.290893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.291482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.291956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.293458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.294297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:20.296644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.298862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 201/408] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:20.304497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.330660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8b480-99cc-4bd0-b593-eb5e854a9545 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.375457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.376636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.377346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.378289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.393925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:20.404639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:20.405549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:20.406295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 208/409] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:20.416085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.477107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf4620-4aa5-4214-93f1-b2f1b2dada07 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.538023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8b79-f001-4279-a839-5ee01a234648 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.570416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:20.572247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.573227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.573550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.573960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.575223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.576668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e1f1d-f151-4546-a3f2-b2878b453f9a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.578664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 202/410] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:20.579900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ab6-b598-48ad-a99b-538c9e768e5c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.580195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ab6-b598-48ad-a99b-538c9e768e5c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.580467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ab6-b598-48ad-a99b-538c9e768e5c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.583846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.585120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.585638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.586003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 203/411] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:20.586493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.586806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179ab6-b598-48ad-a99b-538c9e768e5c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.587793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:20.588300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.590336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8ee4-9b48-4924-bc68-0e97090b3e81 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.601289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:20.604414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 209/412] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:20.611825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.612384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.612860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.636714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:20.639385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:20.725065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 210/413] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:20.726063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.731779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179ab6-b598-48ad-a99b-538c9e768e5c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.788269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.789213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.789498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.789772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/groups/b20ee172ddec429a800b171f92197835/roles/4169efc51b3041918162437c5960c781` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.798422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d56ddd3dc1cf40f7916040db60dc38ab, group_id=b20ee172ddec429a800b171f92197835, role_id=4169efc51b3041918162437c5960c781)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.801299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.826273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 204/414] 10.222.0.74 () {58 vars in 1442 bytes} [Thu Jun 16 16:27:20 2022] PUT /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab/groups/b20ee172ddec429a800b171f92197835/roles/4169efc51b3041918162437c5960c781 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:20.827193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.858850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2dae1-49bf-464e-82ea-a7cbf06fd62b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:20.880433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:20.901510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:20.902837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:20.903297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:20.903715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] PATH_INFO: `/v3/groups/098d615f11ba4b6fba3cab29ae86078f` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:20.905222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] RBAC: Authorizing `identity:get_group(group_id=098d615f11ba4b6fba3cab29ae86078f)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:20.906824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:20.909073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 211/415] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/groups/098d615f11ba4b6fba3cab29ae86078f => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:20.916695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:20.936962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.053599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717c026-a2a5-43f6-a29a-0c90319ad6e1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.074959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0985-f9ba-45ac-a756-4df1e7224678 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.100382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.102114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.103187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.103689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.119382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:21.128942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:21.130155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:21.130926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:21.135148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 205/416] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:21.137256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.137256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.137256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.138083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] PATH_INFO: `/v3/projects/f0df30bd71bf4670afbc831b314be545` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.140727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] RBAC: Authorizing `identity:get_project(project_id=f0df30bd71bf4670afbc831b314be545)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:21.140727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:21.141911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.142629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 212/417] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:20 2022] GET /identity/v3/projects/f0df30bd71bf4670afbc831b314be545 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:21.148404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.215707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.273379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcfe7c-581a-41e0-b3f1-b2f7b342bb46 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.275875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271212d2-1f50-4435-a9f1-8f240f6fd970 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.302897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.314963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.316034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.316539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.317008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.317956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:21.318486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:21.320864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:21.331625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 206/418] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:21.337935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.354435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.359406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.359700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.359981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] PATH_INFO: `/v3/projects/f0df30bd71bf4670afbc831b314be545/groups/098d615f11ba4b6fba3cab29ae86078f/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.362229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] RBAC: Authorizing `identity:create_grant(project_id=f0df30bd71bf4670afbc831b314be545, group_id=098d615f11ba4b6fba3cab29ae86078f, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:21.363837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00351cfb-198b-4ff1-a1b2-f3e33418980b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:21.394595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 213/419] 10.222.0.74 () {58 vars in 1442 bytes} [Thu Jun 16 16:27:21 2022] PUT /identity/v3/projects/f0df30bd71bf4670afbc831b314be545/groups/098d615f11ba4b6fba3cab29ae86078f/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:21.461689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9188a0-8ce6-4cac-873e-79aff37e60f4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.500978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.502137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.502625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.503138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.574227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.589892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:21.591426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:21.617780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6496-2396-419c-93ef-cabeee43d7cc None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.618515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6496-2396-419c-93ef-cabeee43d7cc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.619073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6496-2396-419c-93ef-cabeee43d7cc None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.621399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 214/420] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:21.627213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.627773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.628240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.654955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:21.656609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:21.710627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 207/421] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:21.716528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.751074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 215/422] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:21.770533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.821680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.824039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:21.838093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366be5-8cb5-4b18-8df0-9aacf27998d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:21.903527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:21.905545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:21.908013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:21.908491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:21.909938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:21.913523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:21.915848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 208/423] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:21 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:21.923535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:21.942355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e780d20-f84d-4652-a1b0-bc9d178baa8a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.014768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.015677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.015931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.016227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.045553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:22.054453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:22.055815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:22.056933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:22.060395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 216/424] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:22.066295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.173956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1573ea-238f-453b-a73c-1303d3b3b680 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.181913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccb1c-0273-402d-846a-1d4315e34825 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.184910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:22.223826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.224733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.225045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.225325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.226975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:22.228459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:22.230107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 209/425] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:21 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:22.236661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.258587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.259667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.260146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.260627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.261566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:22.262084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:22.263682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:22.272988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 217/426] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:22.284065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.304724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:22.412904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e595c-7b2e-464c-b82c-1b526b0d88b9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.455390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.456137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.456423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.456702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.458392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:22.458711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:22.460473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af4fc-311a-4510-8a2e-e77595e8b4af None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:22.468353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 210/427] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:22.507685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d049e-2dc3-4e17-aeb1-76498caffe48 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.527585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd205a79-445c-49a5-a3bc-d78ea61c3894 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.528275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd205a79-445c-49a5-a3bc-d78ea61c3894 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.528564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd205a79-445c-49a5-a3bc-d78ea61c3894 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.530479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 211/428] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:22.538668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.539132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.539518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.563839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:22.566137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:22.585018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.585686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.585971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.586249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.648421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 212/429] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:22.662535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:22.675380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.715627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:22.733094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:22.735031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:22.837775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccfa04-10f6-4d21-9c96-04da0a54df35 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:22.848706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 218/430] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:22.857054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.886190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:22.887149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:22.887436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:22.887716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:22.899565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:22.908531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:22.909427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:22.910169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 213/431] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:22.916928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:22.930359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:22.964947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f920af-bfdd-422f-9bbc-11b4ca466db6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.003507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.005225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.005909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.006525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] PATH_INFO: `/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.008826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] RBAC: Authorizing `identity:get_project(project_id=34ed718629b84987b6d5cb7cfbc28c3d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.010493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.012758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 219/432] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/projects/34ed718629b84987b6d5cb7cfbc28c3d => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:23.018223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.060809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.086571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81fe88-ec6e-4532-bdc4-2d7e41256996 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.130222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc25781-af76-4ae2-874c-425b5341f55d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.132329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.133313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.133752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.134253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.135228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:23.135775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.137687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.148026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 214/433] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:27:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:23.153435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.166154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.167270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.167377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.167590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.168879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.170560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.172632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 220/434] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:23.178654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.181682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.267919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b9cb1-bd2b-4745-a8f8-b5fa4d99f35c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.287382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31237df1-4fa9-4159-9d4d-2fda1fe87871 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.306639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.307774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.308412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.308991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.322977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.324727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.325473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.325857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.326761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.327220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=4169efc51b3041918162437c5960c781, scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:23.327559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.329076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.340247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 221/435] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=34ed718629b84987b6d5cb7cfbc28c3d&role.id=4169efc51b3041918162437c5960c781 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:23.396623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.398461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb6c90-19b7-452b-948c-bbafec1a12e0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.511355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 215/436] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:23.516742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.607813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.619108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725eda7-5b28-42b6-9175-48f09a1de31c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.620590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725eda7-5b28-42b6-9175-48f09a1de31c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.621074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725eda7-5b28-42b6-9175-48f09a1de31c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.623680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 216/437] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:23.628846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.629416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.630324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.652607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:23.653824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:23.658950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b675-8d4a-499c-9747-420bb63fff97 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.696999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.705790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.706666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.706947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.707286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.708466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.710009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.712087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 222/438] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:23.717544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.745096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 217/439] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:23.768855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.849503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9301e43-50f4-481d-824d-a63e31e46784 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.894104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.894999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.895409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.895680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.897708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.899154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.899949 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38041241-3176-4424-86a0-9bd8cde07ea4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.901071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 223/440] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:23.908439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.910348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:23.945762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.946610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.946897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.947215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.958196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c379dc78-6b33-4610-98d6-f2083247caf5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:23.966601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:23.967558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:23.968801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 218/441] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:23.969826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c3fe4-0465-43f7-a253-6313ca18f834 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.969826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c3fe4-0465-43f7-a253-6313ca18f834 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.969826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c3fe4-0465-43f7-a253-6313ca18f834 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.970783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 219/442] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:23.973488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c3fe4-0465-43f7-a253-6313ca18f834 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.005551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.023017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286c91-1555-4993-8961-73edbf88ba1b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.057593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.058309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.058596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.058852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.059848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=a0c9d2da81de41eb955b1392d9a2e766, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:24.060160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.061605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a64db2-c7cb-4545-bbba-d92d546d6317 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.067494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 224/443] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=a0c9d2da81de41eb955b1392d9a2e766 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:24.069236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031da23-711d-4313-be96-4853b801efec None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.069236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031da23-711d-4313-be96-4853b801efec None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.069495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031da23-711d-4313-be96-4853b801efec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.087605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b031da23-711d-4313-be96-4853b801efec None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:24.088840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b031da23-711d-4313-be96-4853b801efec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:24.089887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c3fe4-0465-43f7-a253-6313ca18f834 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.136936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.137725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.138017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.138292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.139911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:24.140233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.141703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.155938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 220/444] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:24.157174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.173665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.182520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 225/445] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:24.206052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031da23-711d-4313-be96-4853b801efec None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.269265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.284246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf187b-7889-45b2-b75c-61bf00e33a54 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.323187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.323855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.324141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.324422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.341059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031da23-711d-4313-be96-4853b801efec None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.382967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.383825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.384842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.384842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.401528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:24.408157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.409640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:24.409814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:24.410662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:24.411476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 226/446] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:24.418954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.480019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.507085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.518443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 221/447] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:24.524117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.526359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5b67b-62c3-476c-96df-a86e5f04bf43 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.562105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.563013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.563326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.563603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.564358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:24.564667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.566049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.575009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 227/448] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:24.580542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.629047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e052783-e3ed-4ca7-97c0-e1b7a864d1c7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.665973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.666765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.667075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.667352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] PATH_INFO: `/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.669242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] RBAC: Authorizing `identity:get_project(project_id=d56ddd3dc1cf40f7916040db60dc38ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.670778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.672485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 222/449] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/projects/d56ddd3dc1cf40f7916040db60dc38ab => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:24.678110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.680265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca8c7a-ce73-41d6-8baf-bba3ed78e504 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.697908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.715375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.716033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.716317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.716605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.738511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.788935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0903-8162-4ef0-afca-9331ceaad0cb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.790996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.792509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.826808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.827710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.827996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.828279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] PATH_INFO: `/v3/groups/b20ee172ddec429a800b171f92197835` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.829485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] RBAC: Authorizing `identity:get_group(group_id=b20ee172ddec429a800b171f92197835)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.830935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.832607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 223/450] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/groups/b20ee172ddec429a800b171f92197835 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:24.838741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.896272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 228/451] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:24.902267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:24.920912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.941010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaa865-9a2d-4393-b1dd-4d0963a68b20 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:24.967827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:24.976804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:24.977469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:24.977751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:24.978033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:24.978869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] RBAC: Adding query filter params (group.id=b20ee172ddec429a800b171f92197835, role.id=4169efc51b3041918162437c5960c781, scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:24.979212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:24.980708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:24.986864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 224/452] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/role_assignments?group.id=b20ee172ddec429a800b171f92197835&scope.project.id=d56ddd3dc1cf40f7916040db60dc38ab&role.id=4169efc51b3041918162437c5960c781 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:25.014370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086140-2fc3-4e15-be54-4ee32a769f22 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:25.053117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:25.054042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:25.054316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:25.054586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] PATH_INFO: `/v3/projects/f0df30bd71bf4670afbc831b314be545` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:25.055882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] RBAC: Authorizing `identity:get_project(project_id=f0df30bd71bf4670afbc831b314be545)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:25.057352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:25.059180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 229/453] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:27:24 2022] GET /identity/v3/projects/f0df30bd71bf4670afbc831b314be545 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:25.064927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:25.177160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc440796-03fa-4df1-ab1c-3fe44c0a7205 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:25.216084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:25.216981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:25.217262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:25.217519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] PATH_INFO: `/v3/groups/098d615f11ba4b6fba3cab29ae86078f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:25.218734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] RBAC: Authorizing `identity:get_group(group_id=098d615f11ba4b6fba3cab29ae86078f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:25.220220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98422e48-f8dc-4055-b496-868e1b846866 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:25.221837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 225/454] 10.222.0.74 () {56 vars in 1180 bytes} [Thu Jun 16 16:27:25 2022] GET /identity/v3/groups/098d615f11ba4b6fba3cab29ae86078f => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:25.228034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:25.320458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:25.335598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57a6e-b039-4e2d-99ae-468384cf7d84 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:25.371932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:25.372584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:25.372863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:25.373132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:25.373963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] RBAC: Adding query filter params (group.id=098d615f11ba4b6fba3cab29ae86078f, role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=f0df30bd71bf4670afbc831b314be545) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:25.374273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:25.375742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b6492-2df7-4374-83d6-62ce7e4e7a37 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:25.381602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 230/455] 10.222.0.74 () {56 vars in 1376 bytes} [Thu Jun 16 16:27:25 2022] GET /identity/v3/role_assignments?group.id=098d615f11ba4b6fba3cab29ae86078f&scope.project.id=f0df30bd71bf4670afbc831b314be545&role.id=279eea356bbd45f39595ccaa1656385a => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:28.332725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69d34-d7a4-45fc-ad1c-4631411f276f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.333564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69d34-d7a4-45fc-ad1c-4631411f276f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.333564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a69d34-d7a4-45fc-ad1c-4631411f276f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.334923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 226/456] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:28.340536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158e2d-4aa0-4dc2-9f95-816578c15d26 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.341177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158e2d-4aa0-4dc2-9f95-816578c15d26 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.341177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158e2d-4aa0-4dc2-9f95-816578c15d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.358397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-43158e2d-4aa0-4dc2-9f95-816578c15d26 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:28.359616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-43158e2d-4aa0-4dc2-9f95-816578c15d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:28.428491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 231/457] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:28.453699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69d34-d7a4-45fc-ad1c-4631411f276f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:28.612794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:28.709008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a69d34-d7a4-45fc-ad1c-4631411f276f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:28.786609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:28.788613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.789397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.790156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.819332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:28.822024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:28.862569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ef56-2976-466e-b167-d53b1bacc69b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.863363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ef56-2976-466e-b167-d53b1bacc69b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.863799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ef56-2976-466e-b167-d53b1bacc69b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1fe7c65d-ef9b-4e70-8b66-db940ba136df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:28.866793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:28.867836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:28.868724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:28.868724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:28.868724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:28.868724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:28.870547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 227/458] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:28.871491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 232/459] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:28.873943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea21b1-aacc-4b9a-a779-fcdbf639e6de None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.874436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea21b1-aacc-4b9a-a779-fcdbf639e6de None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.874924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea21b1-aacc-4b9a-a779-fcdbf639e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.878818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985ef56-2976-466e-b167-d53b1bacc69b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:28.896224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-07ea21b1-aacc-4b9a-a779-fcdbf639e6de None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:28.898308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07ea21b1-aacc-4b9a-a779-fcdbf639e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:28.976091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 228/460] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:28.977937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0277b-86a7-4b3f-b6ca-23242109ab73 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.978482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0277b-86a7-4b3f-b6ca-23242109ab73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.978962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0277b-86a7-4b3f-b6ca-23242109ab73 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:28.980696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 229/461] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:28.986610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:28.987161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:28.987651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.006522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:29.008487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:29.052425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.079445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.084650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 230/462] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:29.091074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.106555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985ef56-2976-466e-b167-d53b1bacc69b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.157117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.158550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.159518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.160056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.161337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:29.161875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.164055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.186811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 233/463] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:29.186811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.236335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a867c9-f62b-413d-b4b7-c244776af61b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.282055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.283922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.284424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.284897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.300859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.303693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.309917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c342aaa-44e7-4283-bc2a-e0f097e3aec5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:29.334043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:29.335326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:29.336329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:29.336329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:29.336329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.336329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:29.339273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 231/464] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:29.340316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.360119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.362324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.362766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.363471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.374107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.374760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.377101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.378588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:29.397419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:29.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:29.399183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:29.399183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:29.399183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.399183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:29.399183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 234/465] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:29.399461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.467517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62345128-697d-4b63-883d-9f079826cdb4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.506083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.506767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.507073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.507552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.508266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.509734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253988c9-9000-4109-9861-9654d76a6c75 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.513730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3778a32-8f43-4d54-b83f-3ca88ce57103 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.540021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 232/466] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:29.542393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f8f6-b0db-4f8f-950f-cb953304a66a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.543449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f8f6-b0db-4f8f-950f-cb953304a66a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.543948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f8f6-b0db-4f8f-950f-cb953304a66a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.545630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 233/467] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:29.551652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f8f6-b0db-4f8f-950f-cb953304a66a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.566089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.567214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.567718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.568218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.569948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:29.570513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.572366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c3c94-fd08-4ce7-b4a2-9171957ad291 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.592535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 235/468] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:29.594538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.595105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.595595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.620507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:29.622594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:29.669426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.696385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f8f6-b0db-4f8f-950f-cb953304a66a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.697380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.719473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 236/469] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:29.720300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.751984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.752658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.752942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.753223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.753950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:29.754280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.755699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26fe00-fb48-43c6-9077-cb7fbabc2fa1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.769559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 234/470] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:29.770768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01748f-2a44-4ef2-9701-24889a85cae5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.771106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01748f-2a44-4ef2-9701-24889a85cae5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.771480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01748f-2a44-4ef2-9701-24889a85cae5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.772746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 235/471] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:29.773442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01748f-2a44-4ef2-9701-24889a85cae5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.837998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e21657-b48d-4b91-aed3-10b992213587 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.876131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.876806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.877082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.877357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.878065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.879556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.896549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01748f-2a44-4ef2-9701-24889a85cae5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.900043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 237/472] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:29.900749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.939089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.940173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.940619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.941044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.951216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.957771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.958934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.962850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-cbf3c780-8a90-40ef-b8ba-9efc4fc38bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:29.992127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:29.993041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:29.993875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:29.993875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:29.993875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:29.993875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:29.999643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 236/473] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:30.001938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.002271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.002550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.030315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:30.031634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:30.049802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0098-8b2e-4f71-b579-c8336a587a69 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.095460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.096438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.096861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.097275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.098181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:30.099908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:30.121024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 237/474] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:30.122858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:30.124448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 238/475] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:30.153283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:30.258828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59649715-737b-408c-8a48-885de5ee7b92 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.277531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:30.283009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65856-52b4-4a11-a0d6-c1e44a878aaf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.304484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:30.327426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.328463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.328957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.329407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.330369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:30.332320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:30.334699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.335421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.335735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.336152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.336855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:30.337167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:30.338700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:30.343077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.sql.core [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 16:27:30.343077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 16:27:30.343077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [parameters: {'id': '5800d51d62d54548bdc5d1516225a728', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 16:27:30.343077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80340) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 16:27:30.358572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 238/476] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:27:30.360722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:27:30.361880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application yield resource Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise self.value Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application session.commit() Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 16:27:30.362777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.flush() Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application compat.raise_( Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 16:27:30.363726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:27:30.364625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:27:30.372537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [parameters: {'id': '5800d51d62d54548bdc5d1516225a728', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:30.375569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 16:27:30.377770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.381837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 239/477] 10.222.0.74 () {60 vars in 1131 bytes} [Thu Jun 16 16:27:30 2022] POST /identity/v3/roles => generated 144 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:30.381979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:30.382126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:30.495408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f0cb4-134a-4657-9adb-d4862f4437a9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.503023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe156a-2d64-440d-967e-3444684fe1d2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.552808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.553953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.554336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.554760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.554933 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.555335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.555472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.555786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.556797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:30.558568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec60406-0cb6-43b8-868c-7c3362ba70c0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:30.570149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:30.579027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:30.579962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 16:27:30.580752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:30.583680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 240/478] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:27:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:30.589594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:30.590583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 239/479] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:30 2022] POST /identity/v3/users => generated 249 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:30.627529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:30.712121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32bd6-32e2-4b12-ac93-8d73193c4593 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:30.766039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:30.766945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:30.767380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:30.767872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:30.769447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:30.770098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:30.776272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf74f4-abfa-47c7-883c-7d2b690464a9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:30.786059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 241/480] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:32.101537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08141-8ab5-49e4-9b02-4f4dc7d9c209 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.102098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08141-8ab5-49e4-9b02-4f4dc7d9c209 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.102098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08141-8ab5-49e4-9b02-4f4dc7d9c209 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.103723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 240/481] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:32.109208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.109953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.110398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.136043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:32.137486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:32.233528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 242/482] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:32.281589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08141-8ab5-49e4-9b02-4f4dc7d9c209 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:32.333853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:32.401592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08141-8ab5-49e4-9b02-4f4dc7d9c209 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:32.437564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:32.438432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.438724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.438997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.449994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fcdcf34b-814e-4ae9-9711-3cc0c2084df8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:32.457787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:32.458635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:32.459483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 241/483] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:32.465419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:32.480353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:32.588321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9446f0-8e05-416e-a226-501f987c3aba None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:32.628731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:32.629417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.629704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.629982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.630742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:32.631113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:32.632612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:32.645965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 243/484] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:32.648762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330c81-df4d-4a20-a0e5-a0e9125b270c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.649055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330c81-df4d-4a20-a0e5-a0e9125b270c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.649334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330c81-df4d-4a20-a0e5-a0e9125b270c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.650699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 242/485] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:32.655346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:32.656015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.656418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.656698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.704233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:32.705459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:32.759410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:32.812128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3683b2-9393-48f7-8426-ee4a0f57031a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:32.848198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:32.849008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:32.849298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:32.849577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:32.862755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:32.864301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:32.885536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:32.886451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:32.887857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:32.887857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:32.887857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:32.887857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:32.887857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 244/486] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:32.893004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:32.896799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 243/487] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:32.931846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:32.991490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:32.997691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332c3b5-86bd-4566-a057-42d894253dd0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.025544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.037571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.038617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.039067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.039759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.040642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:33.041190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.042871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ac276-a1bf-474f-b876-68c8f3297635 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.048552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3c3f3-b2a1-4cc2-abcc-b9cb683245a7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.066395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 245/488] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:33.066395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e02c9-2dfa-42d8-8368-b93f1cb78675 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.066395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e02c9-2dfa-42d8-8368-b93f1cb78675 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.066395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e02c9-2dfa-42d8-8368-b93f1cb78675 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.066395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 246/489] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:33.067894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e02c9-2dfa-42d8-8368-b93f1cb78675 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.102373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.103227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.103510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.103784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.114569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4808fee0-7add-44cb-92a3-f26f590d7f23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:33.123167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:33.124101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:33.124877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 244/490] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:33.126974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.127311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.127597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.147677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:33.148871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:33.213349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e02c9-2dfa-42d8-8368-b93f1cb78675 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.229124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 245/491] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:33.230060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.251576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.257871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.260320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.260932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.261413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.270821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.272916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:33.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:33.292754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:33.293647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:33.296593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 247/492] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:33.301960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.346343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e11a9-1a2b-4da3-b5ae-82b9ec3309a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.356481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.382454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.383326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.383672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.383951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.384711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:33.385019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.386404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.395213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 246/493] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:33.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.423713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738e8d-3eea-4e55-a7eb-36a8673ad8a4 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.462723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.463617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.463848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.464125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.476427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:33.482467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-cba333df-9cc5-4cfb-a3bf-362bd7d564e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:33.484301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:33.485223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:33.486001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 248/494] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:33.487324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57532e0c-3568-4755-ba1e-320a381b535e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.487324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57532e0c-3568-4755-ba1e-320a381b535e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.487324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57532e0c-3568-4755-ba1e-320a381b535e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.491186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 249/495] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:33.491186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57532e0c-3568-4755-ba1e-320a381b535e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.512157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791896-3685-4704-9ddd-bf41a8b17233 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.558154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.559002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.559159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.559392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.560259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:33.560571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.562015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.607258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 247/496] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:33.607258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.617276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57532e0c-3568-4755-ba1e-320a381b535e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.620703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.656354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.657123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.657406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.657678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.669541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.671013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e7ed2c70-066e-482e-994f-3171976cc2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:33.687602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:33.688493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:33.689317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:33.689317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:33.689317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:33.689317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:33.689317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 250/497] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:33.689968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaf76a-98e9-4636-9d5d-b7b699daed53 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.690252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaf76a-98e9-4636-9d5d-b7b699daed53 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.690526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaf76a-98e9-4636-9d5d-b7b699daed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.706927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1dfaf76a-98e9-4636-9d5d-b7b699daed53 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:33.708156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1dfaf76a-98e9-4636-9d5d-b7b699daed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:33.718576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f620213-6e16-481f-9787-f8baf90fb75a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.758742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.759531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.759892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.760224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.761164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:33.761522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.763371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.769125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.772737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 248/498] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:33.773563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 251/499] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557eefd-6eb0-4047-9e67-a2f71948b5d0 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557eefd-6eb0-4047-9e67-a2f71948b5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557eefd-6eb0-4047-9e67-a2f71948b5d0 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 252/500] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:33.790656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557eefd-6eb0-4047-9e67-a2f71948b5d0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.877399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.886416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16e3c3-4e34-4e0e-b183-1c3e2b155931 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.893906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557eefd-6eb0-4047-9e67-a2f71948b5d0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.920286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.920936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.921217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.921491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.927254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.928577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.928862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.929182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.929840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:33.930143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.931551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.943159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 253/501] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:33.944220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.991741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:33.994024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.995955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeea58-276b-4fa6-a639-89c03e71402a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.046700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c784-eefe-4d89-8c08-d4d84c3c2546 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.082614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.083437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.083708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.084063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.095136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.096605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.098380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 249/502] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:34.099494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.099853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.100129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.104048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:34.116158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:34.117202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:34.118078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:34.118078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:34.118078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.118078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:34.118307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:34.118307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:34.118759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 254/503] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:34.118759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.184038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 250/504] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:34.184904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.221574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.225300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5aa64-8d07-4147-a3a0-e4209f6bec16 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.258548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.259319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.259602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.259884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.270714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.278707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.293296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9265c-0667-40aa-9076-0b081cde3a42 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:34.296680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:34.297474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:34.298275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:34.298275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:34.298275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.298275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:34.298474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 255/505] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:34.299256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.328141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.328935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.329217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.329487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.336872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.338338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.348524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:34.355418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:34.356502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:34.357583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 251/506] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:34.358318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.407308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332335-8189-44a5-a4c6-3fa3fc5fa981 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.441230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.442050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.442327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.442604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.453796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.453935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.455794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.464844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ed304-173e-4b17-8e72-beccdea24f2d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:34.473365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:34.474186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 256/507] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:34.475786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.503490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.504129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.504411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.504683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.505408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:34.505747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.507545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.519593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 252/508] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:34.520332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.595481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582efd6a-8cfe-4ffb-b887-12cf14dcd83e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.597035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.627950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e132-c1e6-4b97-86df-457abb854edc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.631691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.632499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.632777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.633051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.644290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:34.652328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:34.653270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:34.654024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 257/509] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:34.654660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.669430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.670104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.670389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.670675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.671450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:34.671745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.673146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.685500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 253/510] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:34.686132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.698205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.768918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91be0a-32b8-40c1-8a0c-9a78c7115b8b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.784885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e106c-3856-41a6-bdb8-eee06b40aa29 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.805494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.806467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.806467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.806694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.807563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:34.807887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.809346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.821331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.821987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.822258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.822578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.823334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:34.824076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.825076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.837754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 254/511] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:34.838573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.851593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 258/512] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:34.851593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.872000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.915342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.947925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a50e6-0624-4ee8-bf58-210d54c03057 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.950810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd83dd7-1ce5-40bc-bd4a-f2e312aa390d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.981466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.982245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.982527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.982801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.986336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.987017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.987336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.987610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.988354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:34.988661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.990060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.990827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.992233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.998946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 259/513] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:34.999764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:35.007639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:35.008708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:35.009816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:35.011188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 255/514] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:35.012490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.082117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.101631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87989bb-20e4-4562-ba3c-ca78a698e81f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.117152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b5acb-3988-4017-87e2-51af0a3e80c4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.136842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.137521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.137782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.138090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.138794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.140269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.146171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.153670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.154485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.154772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.155094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.161037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 260/515] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:34 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:35.161994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.162793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.164364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:35.181969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:35.182884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:35.183778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 256/516] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:35.184233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.294363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c86d-1c59-4e6c-9289-13d74f295e08 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.315574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467b111-b8ae-4091-8d8a-92a2b7794f5d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.332999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.333605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.334980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.334980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.343619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.360907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.361719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.362009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.362289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.377074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.378581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:35.397872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:35.398937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:35.400043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:35.400043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:35.400043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:35.400043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:35.400043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 257/517] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:35.400373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.418591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.420080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.455791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.517961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956e09-76db-42a4-bc26-244dd8df0f82 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.541001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 261/518] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:35.541001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.564117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.564759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.565110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.565453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.566488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:35.566838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.568569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.583399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.594417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 258/519] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:35.594417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.650155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d1b4-3ca8-4491-87a6-8a8d1c1996af None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.689024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.689695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.689981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.690238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.691043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:35.691374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.692829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.696092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.700118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f089c2-d299-4a20-ba89-28c88a856b7d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.716978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 262/520] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:35.717904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.739012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.739727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.740012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.740373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.741378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:35.741686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.743093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.755957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 259/521] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:35.757098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.828608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.832906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e897-eb27-4959-8d89-564166eb356a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.863080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18f38-ec24-4ea4-8971-1af1f46aad86 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:35.871079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.872455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.872455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.872455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.883631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.885280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:35.904010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:35.904670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:35.904953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:35.905226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:35.905320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:35.906100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:35.906948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:35.906948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:35.906948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:35.906948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:35.906948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 263/522] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:35.907788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:35.942390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:35.990527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:35.992520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.033733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986b75-c97a-462d-8e92-e751fb89d494 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.069484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.070301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.070592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.071650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.072512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=14ce6a5053f24cff8997908c6d7948a2) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:36.072833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.074293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.091085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 264/523] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/role_assignments?user.id=14ce6a5053f24cff8997908c6d7948a2&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:36.091085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.110052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.123740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 260/524] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:36.124955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.198364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.213700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3274-352e-4e65-80b7-a33f256a2fe8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.251198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16adef-de11-4c4a-b7da-c0043be19564 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.260381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.262327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.262834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.263845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.275375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.277941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.293429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.294315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.294672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.295024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.295924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:36.296308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.297835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:36.298755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:36.299966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:36.300907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:36.308698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 265/525] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:36.308698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.320868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 261/526] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:36.321747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.414308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.445732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11419-6eaa-41b5-8b0d-ef90b013b843 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.465518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de87cfb-13b8-4741-b410-cf2ebfe66944 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.489616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.490617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.491012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.491398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.492427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.505745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.508266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.513338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.514317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.514779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.515258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.516282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:36.517246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.519354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:36.530879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:36.531887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:36.533034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:36.533034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:36.533034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:36.533034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:36.533034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 266/527] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:36.534057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.555328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 262/528] 10.222.0.74 () {56 vars in 1139 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:36.556677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.673026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fe39f-fa9c-4dcc-a9e4-6322609db506 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.701216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.706655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86cd35-693a-4545-82e4-55851f304a7e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.715114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.716819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.717350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.717825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.727284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.729280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.747998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.749140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.750137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:36.750240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:36.751020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:36.751887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:36.753494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:36.754035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.755720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.757047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 267/529] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:36.758023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.772810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 263/530] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:36.774059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.831434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.889939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f594-898d-4310-a1f7-1357935dbb1a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.918274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45d38-9b8a-4981-81c0-115f5c8cba65 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.938477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.939266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.939679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.940019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.967380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.968056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.968345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.968620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.969424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:36.969741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.972325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:36.999410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 264/531] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:37.000362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.023465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.045605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.047111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.148334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.153876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007605-441f-4066-a0f8-f18f99a44bfd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.192766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 268/532] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:37.192766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.200861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.202947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.203528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.204019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.216668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.218540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:37.242467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:37.243599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:37.244405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 265/533] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:37.245866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.326699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.348859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95e09e-43f0-4b4b-b174-f9d523826593 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.395742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.396432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.396721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.396995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.397836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:37.398149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.399697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.402172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832caae-6793-41e5-b966-fce63935ab55 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.411093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 269/534] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:37.411093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.508595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.509344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.509633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.509912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.531459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.533868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d11e77ba-1fdf-4f9e-beca-cb91cf2ec86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:37.563743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:37.564659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:37.565572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:37.565572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:37.565572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:37.565572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:37.566222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 266/535] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:37.567351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f6667-b70e-4af3-b4ca-fee6f1fcb83f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.567639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f6667-b70e-4af3-b4ca-fee6f1fcb83f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.567915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f6667-b70e-4af3-b4ca-fee6f1fcb83f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.570616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 267/536] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:37.580996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.581878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.582364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.604429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:37.604921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb4da0-c6d5-4440-aa06-015778b5ee41 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.606298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:37.621734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.653560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.654237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.654842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.655154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.655967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:37.656435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.658362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.672639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 270/537] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:37.673554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.705164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.707166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 268/538] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:37.708188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.823604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c397a8-075b-44a9-9287-f58310454baf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.847645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d6fb3-ad4b-4802-92f3-bca072267787 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.867313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.868169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.868553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.868929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.869775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:37.870485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.872089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.884994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 271/539] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:37.886036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.898049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.898926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.899343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.899721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.900666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=1a66c447d6e943a29b2bf8a6c5309294) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:37.901080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.902906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.909718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 269/540] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/role_assignments?user.id=1a66c447d6e943a29b2bf8a6c5309294&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:37.910748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.915797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.012305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.053095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29c745-5ee6-45a0-8c7d-5ed8f92653c4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.100922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.101336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef38939-1fd2-4e95-bcf5-7242dc3f0efc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.101720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.102104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.102445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.123859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.125633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:38.152619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:38.153492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:38.154310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:38.154911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 270/541] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:38.155853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.156566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.156943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.157313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.160953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.184543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:38.197362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:38.198327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:38.200037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 272/542] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:38.208862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.328086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fee9fa-8910-4151-a127-7662069dc71b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.360590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.367984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.375353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.376026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.376569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.376850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.377836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:38.378148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.379621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.390378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 273/543] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:38.395530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.426349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ff43-3c9b-4e3b-a4d4-3ec583f3e85a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.463270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.463940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.464221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.464496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.465281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:38.465590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.467035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.480686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 271/544] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:38.486364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.568333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5acda-0dc6-4bd7-bac3-2ef8d378a4f2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.607444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.608308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.608598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.608875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.622017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.623467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.630496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4da585-1757-49ee-b2ad-ba540d2548db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f99a04d8-23fc-487f-af51-5b1ab1880570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:38.642787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:38.643745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:38.644549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:38.644549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:38.644549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:38.644549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:38.645160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 274/545] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:38.646055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a76c0-36b6-4cb9-adc4-0e162e8b84be None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.646343 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a76c0-36b6-4cb9-adc4-0e162e8b84be None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.651594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a76c0-36b6-4cb9-adc4-0e162e8b84be None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.653454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 275/546] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:38.654421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.655547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a76c0-36b6-4cb9-adc4-0e162e8b84be None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.667300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.668508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.668885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.669385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.670905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=7db42ab9d8ff4cd2931f40085f983216) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:38.671500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.673310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a432ffa-5557-4283-83e1-ccf1da865bb2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.680095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 272/547] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/role_assignments?user.id=7db42ab9d8ff4cd2931f40085f983216&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:38.681371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.681659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.681931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.714348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:38.719591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:38.720822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:38.798048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a76c0-36b6-4cb9-adc4-0e162e8b84be None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:38.826447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 273/548] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:38.857527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:38.859315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:38.870294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:38.870812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:38.871196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:38.872152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:38.872465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:38.877290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:38.902383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 276/549] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:38.908099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.009193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e537c-d847-4998-bf30-b468313adb43 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.038431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822ab61-ca98-40de-94b4-52cd993533b2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.041281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.049593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.078004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.079126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.079594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.080044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.088022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.089808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:39.104602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:39.106018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:39.107133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:39.108892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.109797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.110142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.110503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.111805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 277/550] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:39.115398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.151779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:39.159662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:39.160559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:39.161526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:39.165882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 274/551] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:39.166722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.225604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f06d4-1f46-49b1-9029-aa8b70d1badc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.265121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.266203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.268905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.270617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.272174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:39.273545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.275488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:39.310962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 278/552] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:39.316372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.350587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ff77-8e4e-430d-9025-cec7ca54980b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.382246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.394118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.395658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.396150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.396942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.398427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:39.398957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.401128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:39.414064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 275/553] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:39.422560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.490758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68be10d-4b0b-4345-8fd3-316bfe977c32 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.529391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.530055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.530338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.536300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.571427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.622002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.623519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:39.649444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.735266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 279/554] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:39.742401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.771605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd729023-458f-484f-8c5f-0695b046fcb4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.841154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.841965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.842246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.842523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.862281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.863804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:39.899678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:39.900618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:39.901471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:39.901471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:39.901471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:39.901471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:39.910971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 276/555] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:39.912090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:39.937383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a867977-83ea-4fe6-b330-c4f40a8da071 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:39.963757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:39.977245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:39.978054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:39.978336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:39.978611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:39.989890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:39.995369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-fec5bb84-1382-453b-a037-ac0ae0dc078c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:40.013121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:40.014274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:40.015791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:40.015791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:40.015791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:40.015791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:40.017999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 280/556] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:40.019532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef276-8c9c-4f19-b184-e165867d9c67 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.020055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef276-8c9c-4f19-b184-e165867d9c67 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.020505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ef276-8c9c-4f19-b184-e165867d9c67 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.022511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 281/557] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:40.028079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ef276-8c9c-4f19-b184-e165867d9c67 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.042553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.053183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b30fd-b3e3-4ccc-bd9b-86026dc43e05 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.090879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.091603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.091889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.092167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.092895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:40.093204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.094584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2cb88-5b6e-43a3-a789-7b110d600e6d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.107486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 277/558] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:40.108610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.108900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.109180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.134276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:40.135538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:40.168749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ef276-8c9c-4f19-b184-e165867d9c67 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.210764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 278/559] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:40.210764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.226398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.227423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.227711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.227986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.228719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:40.229025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.230417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.252534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 282/560] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:40.258407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.303342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.317739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.338988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d224ee1-67ea-481c-ab2a-4b19418d70d9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.377643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.378460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.378740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.379013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.387017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.388541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.395461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943cca94-d4ab-4dad-886d-5c07d48bf7fb None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:40.406953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:40.408269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 279/561] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:40.409165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.446709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.447996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.448584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.449175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.462831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:40.472548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:40.473532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:40.474397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:40.483019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 283/562] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:40.484607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.542322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103959c4-ecda-476c-bf1e-ecdf67acb4ff None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.580059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.581262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.581547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.581825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.589250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.590713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.600056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:40.605316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:40.606189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:40.607074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 280/563] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:40.607594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.610302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.615999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ec7-251e-4099-a4f8-41af36633399 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.652304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.652994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.653272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.653550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.654393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:40.654709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.656208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f2a8-3325-4703-bfe4-e6319b8bbde6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.679534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 284/564] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:40.679534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab8ed5-fbce-449c-8178-dc69ccd54d5f None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.679534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab8ed5-fbce-449c-8178-dc69ccd54d5f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.679534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab8ed5-fbce-449c-8178-dc69ccd54d5f None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.679926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 285/565] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:40.680650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab8ed5-fbce-449c-8178-dc69ccd54d5f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.730432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5b5ce-6601-48f5-b31e-0bbf04138752 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.769015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.770000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.770463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.770925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.771859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:40.772516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.773907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.786663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 281/566] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:40.787523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.799272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab8ed5-fbce-449c-8178-dc69ccd54d5f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.837707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.838373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.838651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.838922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.839747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:40.840067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.841487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd3e10-9d9f-46ca-814e-f6117954f1ca None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:40.858500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 286/567] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:40.858500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.858500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.858500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.860194 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.873010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:40.874217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:40.881833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:40.905479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b87cf-0579-4a8c-8f83-e2f684a98475 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:40.943482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 287/568] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:40.955329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:40.969461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:40.970573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:40.970862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:40.971169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:40.983428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:40.985572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:41.009970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.010916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 282/569] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.016975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.061254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c09e7-3fe3-4136-b3ab-632747f892f1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.096594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.097392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.097672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.097982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.109294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.110920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.112842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.121465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f974e-ac24-4f7f-a4d3-d442def05ddb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:41.133739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.134648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.135599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:41.135599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:41.135599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:41.135599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:41.135599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 288/570] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.136300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.143374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.162515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.163204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.163490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.163753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.164579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=3fc2a725e2f14b90be5575a030f47dae) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:41.164884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.166325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.171879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 283/571] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/role_assignments?user.id=3fc2a725e2f14b90be5575a030f47dae&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:41.172825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.251339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657a1a7-431f-45cb-9ee2-fa5cd70de45d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.292981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.294134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.294419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.294699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.299632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f423-b2ec-4d2a-b8e0-c2ad321bd7b9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.307330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ec3f72bc-b906-4858-bde6-148b6a426b2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:41.315338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.316324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:41.317140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 289/572] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.318973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c3b3-d6e1-4cc3-bd8a-080796f06781 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.318973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c3b3-d6e1-4cc3-bd8a-080796f06781 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.319621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167c3b3-d6e1-4cc3-bd8a-080796f06781 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.321056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 290/573] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:41.326387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167c3b3-d6e1-4cc3-bd8a-080796f06781 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.353639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.354322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.354605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.354881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.356035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:41.356558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.358277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3b4f7-00ed-4706-995c-0969b838e002 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.371883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 284/574] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:41.378959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.378959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.378959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.397779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:41.399159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:41.407088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.448367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.471775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167c3b3-d6e1-4cc3-bd8a-080796f06781 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.503093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 285/575] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:41.503093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.531082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.532342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.532957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.533415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.534262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:41.534720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.536195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.560608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 291/576] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:41.560608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.623411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf7182-bb3e-484a-ae78-2d511281980b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.664173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.665013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.665395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.665778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.666633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:41.667043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.668562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.677268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df1567-ad4f-4117-896d-d0d886182dd1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.682453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 286/577] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:41.682453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.699567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.713307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.725582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.726520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.726783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.727073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.734431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.735913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:41.751470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.752549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:41.753611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:41.757704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 292/578] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.758881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.788343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c3d4-b4ca-4a00-9403-0e6fa9dfe17a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.825189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.825931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.826219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.826604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.837623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:41.845521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.846414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 287/579] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.847635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.868482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7168a-0f51-461f-853c-541b561d1b3d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.904535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:41.905540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:41.905540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:41.905806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:41.913987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:41.915487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:41.930191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:41.931181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 293/580] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:41.933321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:41.951289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:41.960175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4a765-6e97-4ea1-85bc-5639bd1bcd81 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:41.972947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.000503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.001471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.002074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.002534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.014714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.016227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:42.034825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:42.035748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:42.036615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:42.036615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:42.036615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.036615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:42.036615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 288/581] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:42.038018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.048327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d8553-11fe-4fd9-8a7d-d9a661ac792d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.083381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.084182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.084627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.084971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.085859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:42.086557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.088113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.105389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 294/582] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:42.105389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.158485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5698f-27b3-4023-984a-ac717b042e15 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.195925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.196607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.197163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.197442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.198188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:42.198493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.200269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.206213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.214525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 289/583] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:42.214525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.219524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7554a-5bc6-4b26-8c8d-166044052f19 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.220334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.260520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.261569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.261996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.262433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.263492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:42.263941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.265643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.288845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 295/584] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:42.293717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.323770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6e46-f270-447c-8bed-98e1ca496ff6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.366398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.367107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.367401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.367680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.368417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:42.368828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.370747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.387173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 290/585] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:42.387173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.407106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e42de-4e58-4d1c-bdbf-3f7476f6839a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.442860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.443854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.444128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.444401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/14ce6a5053f24cff8997908c6d7948a2/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.460097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=14ce6a5053f24cff8997908c6d7948a2, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.460842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.461607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.476086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 296/586] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:42 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/14ce6a5053f24cff8997908c6d7948a2/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:42.477183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.480313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.518773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9022b-a5ae-49b4-9f51-0390dd40677a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.577026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.578428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.578864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.579331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.593683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.596067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.617098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc181-ddc8-4302-8179-5932d1dc94ea None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:42.619705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:42.621053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:42.622128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:42.622128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:42.622128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.622128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:42.629566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 291/587] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:42.629566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.667595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.668659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.669031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.669395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.681104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.682576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:42.705095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:42.705997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:42.706844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:42.706844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:42.706844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:42.706844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:42.707508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 297/588] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:42.708203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.759954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4625-414d-433a-9bed-8856e4a94660 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.777060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.778132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:42.801058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.801870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.802495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.802925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.812449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.813924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.828182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e49c90-9308-479d-8836-318bb5948172 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:42.832953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:42.834097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 292/589] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:42.835595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.874849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:42.875677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:42.875974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:42.876477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:42.877623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:42.877944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:42.879634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:42.895145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 298/590] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:42.896547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:42.968594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25f2f9-821d-47bb-98cb-36d6229666d8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.014967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.015923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.016349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.016761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.017620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:43.018141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.019780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.031772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393711e-251e-4d01-afd8-3edd579a0386 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.033001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 293/591] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:43.033977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.047779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.070798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.071663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.072703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.072703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.073291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:43.073694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.075271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.078895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.099012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 299/592] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:43.100145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.148111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c082-e40e-4a75-bfc4-dd19365c9c20 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.184192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.185306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.185726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.186137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.194876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.196633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.204824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c252f0-1dc5-421b-bb14-375a4001ff03 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:43.212002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:43.212920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:43.213832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:43.216185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 294/593] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:43.217181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.241244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.242326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.242326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.242508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.250388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.251955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:43.268022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:43.269022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:43.269862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 300/594] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:43.270950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.308667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.333904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.340474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba70ef6-c214-48de-866a-023d73f8a655 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.377794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.379004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.379528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.380005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.388663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f135e-ef73-45ca-a89c-1dba8f6760e6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.391551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.393119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:43.410913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:43.411860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:43.413499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:43.413499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:43.413499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:43.413499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:43.413499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 295/595] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:43.413768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.429630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.430587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.430587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.430850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.431747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:43.432060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.433500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5e58-512a-412c-aca7-d7825896ee01 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.452022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 301/596] 10.222.0.74 () {56 vars in 1139 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:43.453691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a925a8-4a8d-4d4e-ba00-fb07c71d1ecc None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.454168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a925a8-4a8d-4d4e-ba00-fb07c71d1ecc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.454611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a925a8-4a8d-4d4e-ba00-fb07c71d1ecc None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.456035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 302/597] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:43.457398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a925a8-4a8d-4d4e-ba00-fb07c71d1ecc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.529082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfdf83-63f6-4c5b-ab63-46307d78537d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.555472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.567287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a925a8-4a8d-4d4e-ba00-fb07c71d1ecc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.571993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.572670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.572963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.573241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.574033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:43.574346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.575822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.588647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 296/598] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:43.589544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.602987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.609715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.610525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.610875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.611193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.611932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:43.612239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.613630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc2c6-0774-46c0-a200-596e62bddb0b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.630519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 303/599] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:43.630519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.630519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.630519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.644956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:43.646147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:43.706821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc592a-67c3-49de-bb86-a3602ab6c2ed None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.723153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 304/600] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:43.723153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.753789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.754591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.754844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.755177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.766397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.768212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:43.788957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:43.789834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:43.790683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:43.790683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:43.790683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:43.790683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:43.791579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 297/601] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:43.792665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.824466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.848080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a6a67-95ed-4674-889d-6d75c16cbedd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.858289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:43.888356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.889785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.890192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.890480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.902572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d4657-375b-414b-8924-9b5ffa4ce857 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:43.907991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=b6297dc028a44750a69132257e0aede1, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.909494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.925276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 305/602] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:43 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:43.926663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:43.939811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:43.940646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:43.940931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:43.941209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:43.949887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:43.951501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:43.969095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:43.970042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:43.970897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 298/603] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:43.971969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.085014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d8c02-2a5d-477a-84d7-3f330d448ab7 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.089615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffd6b-0803-490f-a242-a70cf01736c2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.115264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.129632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.132178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.134314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.134602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.134882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.135659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:44.135972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.137378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.139635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.141276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.141750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.142326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.155080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 299/604] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:44.155080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.156250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:44.164212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:44.165202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:44.166033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 306/605] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:44.166705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.270017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997b4a6-6f43-437e-b3d5-b9c8b6bfaf1f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.280645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cb8b1-4935-47c9-93f5-f6dd52467560 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.308368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.309250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.309707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.310046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.311181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:44.311586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.313407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.317873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.318865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.319440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.319440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.327259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.328823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.331833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 300/606] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:44.332733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:44.344542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:44.345446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:44.346329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 307/607] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:44.347585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.386094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.404131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.449392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64c32-f4cd-4984-977e-9c5c040d116a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.459310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9779-0670-48e1-92bd-a962af17f444 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.485654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.486975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.487551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.487997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.488960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:44.489458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.491355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.496618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.497633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.497923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.498204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/1a66c447d6e943a29b2bf8a6c5309294/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.501743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 301/608] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:44.503036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.520796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=1a66c447d6e943a29b2bf8a6c5309294, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.522624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.551862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 308/609] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:44 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/1a66c447d6e943a29b2bf8a6c5309294/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:44.556068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.657768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676490-eec2-43e1-a4b7-414de3e705c3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.661290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.705479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.706592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.707884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.708337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.709364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:44.709875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.711606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159ad12-50eb-41cb-900d-5902600359e2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.733382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 302/610] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:44.733382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921882-b506-4da9-8c87-77a9a14469d6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.733382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921882-b506-4da9-8c87-77a9a14469d6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.733382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921882-b506-4da9-8c87-77a9a14469d6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.733382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 303/611] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:44.736968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921882-b506-4da9-8c87-77a9a14469d6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.745383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:44.748900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3fd06-5c28-454e-b8bf-8712ca69199d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.805743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.806560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.806846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.807575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.818325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.819880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-2861df71-b408-41dc-a47e-d9c67ccffdd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:44.838655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:44.839799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:44.840970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:44.840970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:44.840970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:44.840970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:44.840970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 309/612] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:44.841531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.841806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.842081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.861982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921882-b506-4da9-8c87-77a9a14469d6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:44.867003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:44.868269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:44.898999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:44.900292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:44.900759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:44.901231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/7db42ab9d8ff4cd2931f40085f983216/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:44.917907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=7db42ab9d8ff4cd2931f40085f983216, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:44.919871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:44.943450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 304/613] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:44 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/7db42ab9d8ff4cd2931f40085f983216/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:44.944773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.949890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 310/614] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:44.972437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:44.991632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.088226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960e91-eafc-4350-822d-945c736caba0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.099548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.132756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.133917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.134366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.135279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.147991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:45.156992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:45.157978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:45.158794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 311/615] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:45.165653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.177222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769970-ac9c-4351-9b06-f81d8c076142 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.247753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.248436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.248721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.249119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.250436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:45.250752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.252799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.270750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 305/616] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:45.277637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.321927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac94aa-a32a-435c-9e9a-83ed2287b2f8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.360403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.361083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.361366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.361640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.362428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:45.362738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.364179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.373878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 312/617] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:45.378880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.379408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.384173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.407568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7db9-49af-4489-aa62-6dae6a6b9fd4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.443969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.444889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.445181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.445463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.454195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.455695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:45.470692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:45.471686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:45.472603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 306/618] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:45.480730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.493885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea222f-4397-4826-aea9-e6464c5bd577 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.529080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.529889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.530213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.530633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.541962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.543450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:45.561095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:45.563685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:45.565806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:45.565806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:45.565806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:45.565806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:45.565806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 313/619] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:45.571089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.631878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.667947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2132-9bf6-4d0b-af56-f807705c9160 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.673075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15c05-34a2-42e2-bf9e-08bc6fe63506 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.706978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.707618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.707919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.708198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.708951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:45.709263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.710656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.718629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.719406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.719701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.719981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.720935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:45.721252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.722714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.724058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 314/620] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:45.724863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.729999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.759221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 307/621] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:45.765508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.842609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0976-2efc-4f4d-bb07-fc4980765a19 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:45.860685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:45.879733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:45.880648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:45.880969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:45.881247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:45.888825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:45.890276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:45.909423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:45.910408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:45.911795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 315/622] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:45.916642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:45.987016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4db8ff-3249-44b5-a949-3fee62f3472a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.022057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d642-d862-4ee3-ad40-7e9bb405b758 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.022539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.023384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.023666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.023940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.035344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.036812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:46.073846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:46.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:46.075676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:46.075676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:46.075676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:46.075676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:46.076433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 308/623] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:46.081003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.104600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.105274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.105558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.106042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.108173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:46.108491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.109938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.146437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 316/624] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:46.152690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.167876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.231430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.283441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec7b-76a6-40ac-9b64-9d2c30072784 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.325047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ae4e-0626-4e94-a233-ce296760507e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.369645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.372426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.372712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.372987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.389431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.391310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.391596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.391873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.392615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:46.392923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.394338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.423296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 309/625] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:46.427897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.484896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.486614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449f1fb-0920-4a27-9a24-cd967d33539f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.522373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.558203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f2afc-fad9-4eed-b4b9-b51d5b2df06c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.560762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.595267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.596227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.596611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.596985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.604333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.606284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.613717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 317/626] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:46.615520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9d78d-b1ec-4c0a-a96c-1235ff0bc508 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.616156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9d78d-b1ec-4c0a-a96c-1235ff0bc508 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.616588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9d78d-b1ec-4c0a-a96c-1235ff0bc508 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:46.622565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:46.623683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:46.624548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:46.625116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 310/628] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:46.625425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 318/628] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:46.626840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.627412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.627858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.629756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.649818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:46.651773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:46.739704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29ec36-a198-477e-8aea-95be23f25b3f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.750238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 319/629] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:46.751958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.790214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.791166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.791532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.791874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.807190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.807190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:46.824670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:46.825865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:46.826929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:46.826929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:46.826929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:46.826929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:46.826929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 311/630] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:46.827289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.843479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.846367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:46.889809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8c6be-d0af-47f5-abd8-784515f25d73 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.926870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.927599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.927954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.928326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:46.929130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:46.929446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:46.930897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:46.940235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b273ac8-cc7f-4a2f-83bc-6d4dc12a4f83 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:46.947088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 320/631] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:46.947088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:46.982448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:46.983441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:46.983849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:46.984302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.002515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5faf3fa7-b7a2-4819-81e2-4bc636656e80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:47.011784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:47.012801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:47.013602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 312/632] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:47.014790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82f5b-685e-475b-90fe-dcf7c0efc578 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.014790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82f5b-685e-475b-90fe-dcf7c0efc578 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.014957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82f5b-685e-475b-90fe-dcf7c0efc578 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.019119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 313/633] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:47.019119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e82f5b-685e-475b-90fe-dcf7c0efc578 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.063142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb874f7-6a26-443d-9944-f1a1d3833336 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.092114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.103320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.107413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.108070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.108358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.108640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.109387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:47.109685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.111175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.124987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 321/634] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:47.125732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.136311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e82f5b-685e-475b-90fe-dcf7c0efc578 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.176249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.177497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.177815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.178107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/3fc2a725e2f14b90be5575a030f47dae/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.193398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=3fc2a725e2f14b90be5575a030f47dae, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.195280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d14d3-c998-4fe8-8335-2043325d2653 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.213941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 314/635] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:27:47 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/3fc2a725e2f14b90be5575a030f47dae/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:47.215887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.216529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.217337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.238709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a5b1c-03f6-4b93-9949-8043d414b244 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.243507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:47.245418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:47.282680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.283467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.283750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.284019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.284963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:47.285273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.287325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.297282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 322/636] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:47.298202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.359570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 315/637] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:47.359570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.361738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.435980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68378718-be62-44b8-919b-0cdf429204e4 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.438313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.480942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.481758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.482042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.482323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.491948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6bfd3-c9ee-41ba-a6c9-d2fafac1f03e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.492331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.494119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:47.514781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:47.515788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:47.516629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 323/638] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:47.517755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.542046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.543691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.544188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.544654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.560743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.563588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:47.589667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:47.590729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:47.591694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:47.591694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:47.591694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:47.591694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:47.592746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 316/639] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:47.593462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.641131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f550e5-38ce-4893-96b0-5a178f4bcac9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.649931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.681464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.682646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.683187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.683748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.695480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:47.703821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:47.704788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:47.705615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:47.706235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907f49-8ecd-438b-974d-c6dbfac873de None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.708528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 324/640] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:47.710589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.719762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.753272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.754103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.754428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.754962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.757029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:47.757361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.758897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.775171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 317/641] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:47.775905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.830106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4657a88-5192-4897-9a0b-dd3a17159cd6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.868381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.869848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.870710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.871188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.872239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:47.872962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.874495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.886907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af52be-0f93-432a-b680-2c3e2b3c0572 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:47.895186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 325/642] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:47.896375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.920919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:47.921706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.921989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.922443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.923010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:47.923363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:47.924756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96a963-360a-40dc-9d0a-0a72fff7ee32 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:47.925189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.934049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 318/643] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:47.935427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d544-25c4-4e6d-ba42-852f89cf55b5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:47.935672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d544-25c4-4e6d-ba42-852f89cf55b5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:47.935979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d544-25c4-4e6d-ba42-852f89cf55b5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:47.939350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 319/644] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:47.939977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2d544-25c4-4e6d-ba42-852f89cf55b5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:47.979640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.012631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ad39d-cbab-4395-a453-3a20b88c8461 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.049414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.050237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.050547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.050913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.051808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=14ce6a5053f24cff8997908c6d7948a2) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:48.052126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.054522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.060754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2d544-25c4-4e6d-ba42-852f89cf55b5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.063196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 326/645] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/role_assignments?user.id=14ce6a5053f24cff8997908c6d7948a2&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:48.064024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.110247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.110509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.110722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.110955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.121165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.122645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-49db5dc4-8534-411b-b3da-693d473dd55e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:48.141903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:48.142885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:48.143988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 320/646] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:27:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:48.145082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.145082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.145359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.166464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:48.168336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:48.178380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.191658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722489c2-e43e-4d8b-b6f1-fac7bba416ed None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.231793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.232984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.233467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.234019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.246906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.248753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:48.268765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:48.269722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:48.270576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:48.270576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:48.270576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:48.270576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:48.270797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 321/647] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:48.270797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.272989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 327/648] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:48.278505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.294460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.396453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0ad18-2384-4c8c-bda2-763b647f7759 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.403161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5e7fc-3243-4058-9413-c132d07fa8ed None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.439206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.440211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.440671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.441113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.442023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:48.442339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.443904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.445388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.446071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.446358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.446629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.447393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:48.447703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.449074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.463369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 328/649] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:48.464242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.469587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 322/650] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:48.470943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.471662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.577566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.601735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f5bd1-b6ba-4aef-b3f9-01fba79f3f92 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.602246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e66dbf-1a71-4bf7-a8af-55ec2e9e3ff8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.638974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.639950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.640320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.640692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.648223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.649876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.650004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.650409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.650821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.652136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.664348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:48.668925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:48.670014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:27:48.671134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:48.675264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:48.676136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:48.676923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:48.677521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 323/651] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:48.677726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.679807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 329/652] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:48.681005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.754391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.797427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d6f0d-df18-4a20-baa1-5c9bf9ac737d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.812285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6824c-4d5c-48fd-a9e0-2ea6172cb3b4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:48.842813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.844124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.844852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.845713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.847040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:48.847808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.850298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:48.853911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:48.854052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:48.854264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:48.854543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:48.863688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:48.874692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 330/653] 10.222.0.74 () {56 vars in 1137 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:48.875840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:48.936552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:48.938033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.008416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4f744-3579-4686-96fa-ccc9ce9f6494 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.041204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.061783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.062854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.063384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.063857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.064029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 324/654] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:49.064872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.065231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:49.065748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.067653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.080757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 331/655] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:49.082211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.110039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.195646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf21c8-1f18-4fee-9a3e-7f15474a2199 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.214335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea5474-5233-43b2-a9f2-7ce05d34c74b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.231567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.232158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.232450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.232728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.259597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.261478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.261962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.262642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.277059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.278410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:49.296685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:49.297866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:49.298734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:49.298734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:49.298734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.298734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:49.298734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 332/656] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:49.299505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.324371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.328553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.338259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.376497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.433016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e86dfe-a31d-4a49-93c4-81a3ca6cc681 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.460652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 325/657] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:49.461860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.471839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.472649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.472931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.473332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.485174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.486634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-35307be4-63c8-4ee1-9a7b-2b60ac1f0851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:49.505031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:49.506049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 333/658] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42681b-c1ce-4299-b289-262acf23db7d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42681b-c1ce-4299-b289-262acf23db7d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42681b-c1ce-4299-b289-262acf23db7d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.512137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 334/659] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:49.512642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42681b-c1ce-4299-b289-262acf23db7d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.582456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7416b5-3194-4226-b4ce-8ced52610b89 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.601905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.619389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42681b-c1ce-4299-b289-262acf23db7d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.621963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.622710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.623081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.623435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.624231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:49.624615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.626092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.634547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.645498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 326/660] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:49.646432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.654517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.655410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.655699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.655975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.667500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.669128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9fe5a50c-82dc-4724-8df5-034a36ede079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:49.686618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:49.693359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 335/661] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.694238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.706627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:49.708007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:49.758643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff886c-02fb-4566-a36f-4ec18cce8143 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.781095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 336/662] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:49.782430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.803184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.803992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.804299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.804573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.805300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:49.805658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.807138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.819751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 327/663] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:49.820570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.868218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.884383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:49.908675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ecf9-7a58-4c44-bbe8-28394be63871 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.929238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0254f-497d-469c-881f-02b26d808753 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:49.952680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.953482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.953765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.954086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.961056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.962515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.964798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:49.965477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:49.965755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:49.966035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:49.966763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:49.967092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:49.968468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:49.977057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:49.978114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:49.978997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 337/664] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:49.979532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:49.992980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 328/665] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:49.992980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.092789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88f9b1-c17d-4675-9bb7-dc27df39a627 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.102811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383312f9-9ec1-464c-be6f-18f45613f9d9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.121104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.128355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.138023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.138835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.139150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.139429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.140558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.141807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.142584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.143032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.150365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:50.152129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.153853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:50.157998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:50.158958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:50.159792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 338/666] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:50.160451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:50.174545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:50.175585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 16:27:50.176506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:50.179205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 329/667] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:50.180670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.281097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac7c44-ff1c-4a8a-8004-189c2b389ca7 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.283472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99422fdf-a4ea-454c-b424-290a97eda08e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.319336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.320152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.320437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.320705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.320820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.321369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.321656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.321929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.322716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:50.323022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.324483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.328698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.330184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.338474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 339/668] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:50.340170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:50.346388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:50.347525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 330/669] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:50.348456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.393945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.404099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.455728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23d629-5211-4d07-bd78-fbe4cfdf3610 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.457859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c636c8-b775-407f-8a40-378b71256d53 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.491946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.492631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.492916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.493060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.493168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.493734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.493924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:50.494016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.494234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.494326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.495452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:50.495770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.495954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.497187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.503854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 340/670] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:50.504634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.515031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 331/671] 10.222.0.74 () {56 vars in 1139 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:50.515764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.610851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.615978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e290b3-6ff7-4ff0-bb75-b37bc33e599d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.617414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87df2b5-8e64-4063-a836-5ce8d314e327 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.628789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.652973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.653639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.653791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.653939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.654172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.654458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.654744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.655301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:50.655404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.655490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.655876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:50.656189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.656701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.657620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6482-a240-45b5-8a37-1f7139836e4c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.664472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 341/672] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:50.665727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc996f3-2f17-4cd9-9d24-df37891b0477 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.666017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc996f3-2f17-4cd9-9d24-df37891b0477 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.666296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc996f3-2f17-4cd9-9d24-df37891b0477 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.667559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 342/673] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:50.668271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc996f3-2f17-4cd9-9d24-df37891b0477 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.669550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 332/674] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:50.670555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:50.805015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc996f3-2f17-4cd9-9d24-df37891b0477 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.812686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6175fd6-cf1c-4262-9aa0-c48fc0eedaa9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:50.848595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.850341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.850341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.850341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.861776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.862012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:50.862678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.862964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.863268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.863362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ae3ca938-65f5-4ffd-a304-612395c5e7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:50.881588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:50.882566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:50.883803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:50.883803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:50.883803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:50.883803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:50.883803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 343/675] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:50.884980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:50.886440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:50.886440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:50.901127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.906059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:50.907380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:50.929627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:50.954942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:50.956565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:50.980350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 344/676] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:50.981261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.089891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 333/677] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:51.089891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.094015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c033ab5-6053-4b75-a62e-644f7b381a17 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.132666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.133321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.133589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.133940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.135231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=1a66c447d6e943a29b2bf8a6c5309294) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:51.135553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.137012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.155101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 345/678] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:50 2022] GET /identity/v3/role_assignments?user.id=1a66c447d6e943a29b2bf8a6c5309294&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:51.155101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.168272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.182758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.220790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92183db2-4651-4386-a658-0f5e64bec966 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.260298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.261002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.261276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.261552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.262296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:51.262679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.264354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.278001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 334/679] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:51.278888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.300024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e1d42-1e70-4e90-9071-85a8a737866c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.352650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.353469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.353756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.354033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.371641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.372506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:51.396189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:51.397149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:51.398137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:51.398137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:51.398137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:51.398137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:51.398137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 346/680] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:51.399293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.416436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9acc-f9b6-4c7d-92e5-949b790925bd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.443090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.465356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.466466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.466760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.467318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.480419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.482077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.488438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:51.502408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:51.503613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:51.504718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:51.504718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:51.504718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:51.504718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:51.511297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 335/681] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:51.512437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.528338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3763f4a-264b-4ec8-823a-55cefab13feb None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.573812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.575158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.575596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.576426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.586704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.589310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:51.610217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:51.611537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:51.612685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:51.615594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 347/682] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:51.616949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.634639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5ee59-9b3a-4085-90de-5260da5ecf29 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.680939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.682419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.682955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.683578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.684753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:51.685625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.687843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.705850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 336/683] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:51.707432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.739039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c140d-6025-451d-b1a1-19eca8184769 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.754461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.774627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:51.782642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.783983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.784378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.784783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.786093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:51.786541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.789314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.801712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 348/684] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:51.808138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.853286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedbf70-9faf-4ae1-8455-a1306ea9934e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.900071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.900929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.901337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.901856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.902831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:51.903290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.904952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.921831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5ff4b-f592-4dda-9fa1-d7055d50a958 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:51.926116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 337/685] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:51.927120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:51.959651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:51.960829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:51.961325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:51.961783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:51.962771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:51.964542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:51.990354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 349/686] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:27:51 2022] POST /identity/v3/users => generated 254 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:51.991726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:52.030278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.064456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.066913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab6a9e-6d4b-4036-98a3-1629179581bc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.157309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.158136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.158420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.158696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.177059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.178952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.182351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce5d75-afa4-4054-a5da-cf54e36d6f09 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:52.203587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:52.204514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:52.205377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 338/687] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:52.218776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:52.231735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.232497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.233071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.233247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.317913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.319609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.351485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d43b-40fc-4931-8c1e-e2edaa3c78c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.381443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.394761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.395679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.396031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.396424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.397287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:52.397666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.399555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.413082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 339/688] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:52.419536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:52.429139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.448182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 350/689] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:52.454267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:52.549333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322f76d-842a-421b-a024-789dc48f3d03 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.589255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.589933 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.590217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.590490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.591363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=7db42ab9d8ff4cd2931f40085f983216) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:52.591677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.594327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.611008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 340/690] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:52 2022] GET /identity/v3/role_assignments?user.id=7db42ab9d8ff4cd2931f40085f983216&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:52.667319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00b58-3400-430a-9eb9-893bb3127cf5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.721800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.722631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.722914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.723215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.739246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.740808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.754930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-2820ca6d-123e-4480-8b66-083b4ad5d9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:52.766310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:52.767286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:52.768136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:52.768136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:52.768136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:52.768136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:52.768136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 351/691] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:52.773727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:52.875301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:52.927119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1a3a3-e6b0-4295-a1cc-05a5a4e0e7db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:52.963700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:52.964379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.964675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.964955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.965681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:52.966002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:52.967409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:52.968293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31150-33f2-4a40-af94-2e1ce109f023 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.968584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31150-33f2-4a40-af94-2e1ce109f023 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.968989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c31150-33f2-4a40-af94-2e1ce109f023 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.970224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 352/692] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:52.976220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:52.976485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:52.976773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:52.981168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 341/693] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:52.986656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.004795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:53.007408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:53.115270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 353/694] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:53.130169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fea585-1dfd-4f58-9aac-06d4fbc683f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.148382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.171989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:53.184830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.185663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.185951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.186226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.194545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:53.196200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:53.213296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:53.214240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:53.215305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 342/695] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:53.221956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.260900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bdb0e-9a99-4b7f-9a31-8f82f3122f2c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.292983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:53.300057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.301226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.301722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.302197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.314125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:53.322463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:53.323434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:53.324244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:53.331168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 354/696] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:53.332977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.348435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228492c5-6493-42fb-bd4f-1c2ad699028d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.394282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.395016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.395335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.395617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.396484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:53.396814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:53.398560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:53.414471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 343/697] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:53.421957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.513216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676f72c-2b05-4b93-b21f-e858b034f451 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.528714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:53.587241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055afc0-f416-4805-bc91-756413a149da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.588326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055afc0-f416-4805-bc91-756413a149da None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.588810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055afc0-f416-4805-bc91-756413a149da None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.589286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055afc0-f416-4805-bc91-756413a149da None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.590275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055afc0-f416-4805-bc91-756413a149da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:53.590382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ff44a-7524-4e91-ac26-285c5db4a2a5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.590888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055afc0-f416-4805-bc91-756413a149da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:53.592494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055afc0-f416-4805-bc91-756413a149da None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:53.621907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 355/698] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:53.632157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055afc0-f416-4805-bc91-756413a149da None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.633676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.634366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.634656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.634934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.635845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=3fc2a725e2f14b90be5575a030f47dae) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:53.636166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:53.638019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236af93-c735-4ded-b0a2-9fa79a1e3fb0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:53.646380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 344/699] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/role_assignments?user.id=3fc2a725e2f14b90be5575a030f47dae&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:53.701073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:53.719727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be981-683c-4a60-b85e-327df539822b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.719982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be981-683c-4a60-b85e-327df539822b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.720171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10be981-683c-4a60-b85e-327df539822b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.721827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 345/700] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:53.728416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.728702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.728980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.760680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055afc0-f416-4805-bc91-756413a149da None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:53.768231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:53.770079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:53.801223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:53.802342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:53.802924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:53.803862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:53.817622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:53.819631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:53.839081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:53.840022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:53.840885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:53.840885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:53.840885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:53.840885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:53.841097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 356/701] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:53.846362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.882265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 346/702] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:53.909499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:53.964419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.003698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87200c1b-69a4-4268-af1f-ad1ec21bd5be None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.007312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.026303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63e9e6-0f47-41c3-bbad-7d86af8a1a29 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.048517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.049192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.049902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.050190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.050927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:54.051276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.053242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.065913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.066865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.067276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.067702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.068491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.068866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 357/703] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:54.072521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.076204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:54.095245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:54.096336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:54.101332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 347/704] 10.222.0.74 () {56 vars in 1129 bytes} [Thu Jun 16 16:27:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:54.105160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:54.222681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc07884-bbfa-45b1-8a3c-798cb4bfe120 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.238455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab53b64-4ebf-4474-95f1-1f67950d905d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.249877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.261917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.262729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.263015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.263321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.271736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.273317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.281601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.282657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.283303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.283806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.284745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:54.285267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.287006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:54.297139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:54.298121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:54.299530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 358/706] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:54.300514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 348/706] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:54.305317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:54.305629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:54.372187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.449003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f611a-ee3c-41d3-874d-1063ee73f7ff None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.491507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.492957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.493654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.494236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.495969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:54.496490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.498396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.523786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 349/707] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:54.529552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:54.540900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd451a4a-7965-44a1-b6ec-72a37fb12c63 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.591553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.617477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.617901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.618184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.618455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.619201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:54.619570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.621258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d93486-7c83-4613-bbe0-a1e88bed954f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.642475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 359/708] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:54.674402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f220f4-b14b-463f-bac0-8bbae02bc1fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:54.739969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:54.741041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.741607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.742098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.828160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:54.830185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:54.879449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:54.907784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11ff1-4503-4bed-8221-015bcaa3e32b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.908356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11ff1-4503-4bed-8221-015bcaa3e32b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.908356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d11ff1-4503-4bed-8221-015bcaa3e32b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.910043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 360/709] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:54.916019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:54.916341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:54.916595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:54.953031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:54.954294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:54.967591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 350/710] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:54.973399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.035632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 361/711] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 16:27:55.061267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.082658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.089973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a1e8-bd73-4e1c-b5da-27295837513f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.123600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.130153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.131776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.132317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.132943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.144934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.146944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:55.167837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:55.168828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:55.169766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:55.169766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:55.169766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:55.169766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:55.172216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 351/712] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:55.181747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.198026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43300058-4bb4-4024-8ea2-1a9e825b95ba None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.232917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.233733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.234018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.234298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.249694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:55.257239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:55.258886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:27:55.259798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:55.268576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 362/713] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:55.272866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.380220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a4e2-19fb-48ec-a704-ec09a782b2d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.427622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.428448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.428736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.429014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.429756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:55.430065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.431585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.452397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.462887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 352/714] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:55.485878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.501570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6a6c7-418f-49c1-8cf9-b296b03a3a10 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.553426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.554672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.555950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.556599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.558616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:55.559959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.561793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.577082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 363/715] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:55.582505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.587880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.652171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771c83a-3308-4d35-84b0-3de629e1759d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.689275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.690107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.690629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.690739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.697164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfd18b-0a39-477f-bd23-7678386c474f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.698522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.700588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7770cd83-1ed5-4e8f-88e1-766fce7292ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:55.716591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:55.717621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:55.718516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:55.722727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 353/716] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:55.724225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c8c9-d1ad-4b36-9ade-7c4b30cdabbc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.724633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c8c9-d1ad-4b36-9ade-7c4b30cdabbc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.725515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c8c9-d1ad-4b36-9ade-7c4b30cdabbc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.731392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 354/717] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:55.732255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c8c9-d1ad-4b36-9ade-7c4b30cdabbc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.739780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.741104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.741614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.742093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.752555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.755195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.757122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-90fbe142-11de-4057-8fc3-bcdb2a10c715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:55.777440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:55.778391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:55.779446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:55.779446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:55.779446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:55.779446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:55.779446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 364/718] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:55.781000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.781351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.781711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.800914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:55.802791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:55.853166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c8c9-d1ad-4b36-9ade-7c4b30cdabbc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:55.855108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:55.887084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 365/719] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:55.887084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.913283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:55.914310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:55.914835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:55.915335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:55.916356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:55.916884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:55.918608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:55.940330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 355/720] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:55.941554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:55.993612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae673c-a509-4ca7-aebe-0c4ff58cea7b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.019448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.034203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.034755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.035060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.035358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.036093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:56.036401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.038071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.050912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 366/721] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:56.051769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.052601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0548a5-8ebd-41df-bedf-6ac6e528b3b8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.090174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.091547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.092044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.092523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] PATH_INFO: `/v3/services/network` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.093406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.094983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:56.108028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:56.109081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:27:56.109899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:56.117988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 356/722] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:55 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:56.119139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.149958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.164282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18375d60-75ae-47ed-8332-c509fee69b39 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.201868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.202320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.202613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.202892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.203838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:56.204146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.205704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.219099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 367/723] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:56.219099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.248846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec89ff-701b-41ba-bf7b-0199e97f5015 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.270526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.322671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.325963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.326283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.326522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.349651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.351410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:56.375196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:56.376241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:56.377777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 357/724] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:56.382613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fef8c-fe14-4b45-8780-fd50411c8981 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.392230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.429126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.430351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.430833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.431689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.432904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] RBAC: Adding query filter params (name=network) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:56.433552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.435790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc877f5-0e14-44b7-9eb3-7eaf5d6e64ba None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.447219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 368/725] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:56.447219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216679a5-0f51-4f6e-b420-99531a0cdd7b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.447219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216679a5-0f51-4f6e-b420-99531a0cdd7b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.447219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216679a5-0f51-4f6e-b420-99531a0cdd7b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.450929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 369/726] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:56.451616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216679a5-0f51-4f6e-b420-99531a0cdd7b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.511298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.530953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fd06-ad3c-48f6-adca-70122c661767 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.574478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.575678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216679a5-0f51-4f6e-b420-99531a0cdd7b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.575815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.576260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.576689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.577841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:56.578297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.580535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c80a40-1130-4a5d-8acf-fc03c12dd94d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.590331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.608013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 358/727] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:56.609869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.610367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.610868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.626314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.627033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.627344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.627622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.628393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] RBAC: Adding query filter params (type=network) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:56.628701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.629937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:56.630180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.631859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:56.640852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 370/728] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:56.640852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.761475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240322d-8665-460b-a920-550b0a9a42bc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:56.784855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 359/729] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:56.798857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:56.799936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:56.800427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:56.800898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:56.840517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:56.883472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.901121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:56.902720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:56.933315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:56.963146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef0c8e-e8ef-4595-9655-fe21c1b8f119 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.004504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.005959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.007565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.008931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.031801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:57.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:57.041096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:57.041980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:57.044866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 360/730] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:57.051111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.134880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 371/731] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:27:57.140601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.235085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5b147-2efa-4311-8cdd-5b6644a1b4c9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.283535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.301372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.302047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.302330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.302767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.305070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:57.305390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.306804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.308480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.318396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 361/732] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:57.324931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.359460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1e89-0e3f-482b-8f86-23b14a6c04f9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.425338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.426139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.426423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.426688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.437649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.439115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.454012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b88645-e424-4e41-8030-8bb6389bc69d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-5a565c53-7233-456a-86d3-0264b22cc0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:57.457378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:57.458321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:57.460141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:57.460141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:57.460141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:57.460141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:57.460141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 372/733] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:57.465481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208098a-80f5-47a0-a5c3-5627c14ba4d5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.465481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208098a-80f5-47a0-a5c3-5627c14ba4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.465481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208098a-80f5-47a0-a5c3-5627c14ba4d5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.465481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 373/734] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:57.466082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208098a-80f5-47a0-a5c3-5627c14ba4d5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.509611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.510239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.510518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.510813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.532093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.533981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4937d3eb-e0c9-4aeb-a754-c2899a786a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:57.560552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:57.561470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:57.562322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:57.562322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:57.562322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:57.562322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:57.564575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 362/735] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:57.569645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.569645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.569645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.592964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:57.595209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:57.600732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208098a-80f5-47a0-a5c3-5627c14ba4d5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.629657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.639793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.640567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.640802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.641079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.641868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:57.642169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.643647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.658247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.661175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 374/736] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:57.662517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.685724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 363/737] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:57.686504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.788539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb883d0-e0e6-43f3-8603-485a5ac0bf95 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.796490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c0b37-4b3d-4131-b24d-9464623bce93 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:57.835615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.836645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.837042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.837481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.838246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:57.838418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:57.838848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.839607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:57.840086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:57.840457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.840735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:57.849818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:57.852536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:57.859417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 375/738] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:57.860509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:57.875020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:57.875969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:57.876829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:57.879130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 364/739] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:57.880143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:57.920876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.938462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:57.978786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab9e02-18e0-426e-afa5-062ab700ba94 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.009598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f545b6-aa37-48c4-b32e-a33e037fa82a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.018397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.019494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.020004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.020517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.021556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.023170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.043256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 376/740] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:27:57 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:27:58.045030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.055030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.055948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.056233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.056522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.076233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.077648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:27:58.104130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:58.105237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:27:58.106275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:58.111670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 365/741] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:58.125795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.187120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca2139-95fb-4b61-b43c-945b3e6a7e52 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.196969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.230830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.231972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.232482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.233140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.234313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:58.234868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.236721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.251786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 377/742] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:58.258779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.277105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.287359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f2b99-c66b-4a82-b043-012388460f2d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.324884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.325493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.325785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.326051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.327003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:58.327351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.328797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.349089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 366/743] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:58.358487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.381748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de79bef-82b2-4595-86d1-e5f679ac1c05 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.420032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.421065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.421653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.422153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.423264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=687bbc1e0d174b78a51d01542d1fd03a) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:58.423780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.425957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369544f-39f4-4b95-bdd3-1f14325d0571 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.433461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 378/744] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/role_assignments?user.id=687bbc1e0d174b78a51d01542d1fd03a&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:58.467631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b8401-a5a7-47d2-9c07-8abb3409fa1a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.507935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.508631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.508914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.509203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.549264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.584517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179b177-28a7-4a3a-8363-908e13ff1667 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.585326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179b177-28a7-4a3a-8363-908e13ff1667 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.585816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179b177-28a7-4a3a-8363-908e13ff1667 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.587716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 379/745] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:27:58.598137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.598961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.599573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:58.640261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:58.641495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:58.651378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:58.653065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:58.680844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.766731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 380/746] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:58.797510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 367/747] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 16:27:58.803163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.806862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:58.919542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:58.938821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213c7ef-7cb6-4eae-89fd-336fde5a7d53 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.982057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc6cdb-a3a6-4bfc-a650-47264d448c9c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:58.985117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:58.985931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:58.986216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:58.986494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.009720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.011245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:59.029980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:59.030921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:59.031827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:59.031827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:59.031827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:59.031827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:59.031827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 381/748] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:59.036900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.052454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.059791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.060623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.060903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.061174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.092006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:27:59.107789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:59.110068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:59.112199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 368/749] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:27:59.119095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.143478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66d05e-3c98-4ce9-994e-e48ebb0dcd93 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.180010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.180602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.180886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.181168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.181899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:59.182211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.183655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.206448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 382/750] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:27:59.213511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.291956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.326366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4a34b-2212-4caf-a9e5-397562bffa2d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.367841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.368674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.368960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.369238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.378171 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.379853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:27:59.396016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:59.397037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:27:59.397966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 383/751] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:59.405404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.438875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06a104-2702-417f-af6c-84bd8543684c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.472604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.476773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.478001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.478492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.479167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.480403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:59.481496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.483451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1befa75-111a-4858-ba4b-c08cd5f72720 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.499025 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 369/752] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:59.501340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d701d-281a-4987-88e1-1b9252469456 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.501936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d701d-281a-4987-88e1-1b9252469456 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.502395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d701d-281a-4987-88e1-1b9252469456 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.508352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 370/753] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:27:59.508352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d701d-281a-4987-88e1-1b9252469456 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.529477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabebce3-3ece-4530-8656-d33fc522f135 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.570394 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.578156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.578949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.579332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.579680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.580706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:59.581089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.582915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe48762-8bb7-44ac-bbfe-d456489a3905 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.597900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 384/754] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:59.604105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.604396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.604680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.627604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:59.629444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:59.648902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d701d-281a-4987-88e1-1b9252469456 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.695987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.696761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.697047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.697326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.712931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.714397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.727088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 385/755] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:27:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:27:59.727088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:27:59.747700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:27:59.748760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:27:59.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:27:59.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:27:59.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:27:59.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:27:59.749631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 371/756] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:27:59.757575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.795126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.844660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18462-3b9c-4dd4-9584-c64bc9f23110 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.870930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:59.883880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.884548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:59.884835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:59.885115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:59.886014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=3fc2a725e2f14b90be5575a030f47dae) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:27:59.886260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:59.887764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:59.899174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 386/757] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/role_assignments?user.id=3fc2a725e2f14b90be5575a030f47dae&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:27:59.901437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:59.907546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7f2e-16e5-4698-83f6-60610de2e0be None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:59.996270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:59.998015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.000623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.001152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.002131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:00.003887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:00.031895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 372/758] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:27:59 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:00.108793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee0bcc-9677-49a7-ae41-bbd0ceb617f6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:00.164951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:00.166047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.166569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.167092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.168134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:00.168648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:00.170266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:00.185057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 387/759] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:00.192192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:00.323500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:00.343260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc1517-d8e2-4c8b-bff2-bbf02c6ededf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:00.381302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:00.382067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.382345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.382620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.390839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:00.392362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:00.407765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:00.408716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:00.409620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 373/760] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:28:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:00.415044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:00.465423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:00.533004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aca57-b867-407f-a8ce-4e617a7e968e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:00.571909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:00.572592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.572870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.573140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.574046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:00.574296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:00.576004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f3030-f691-4a1d-8181-2816b57faa49 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:00.597372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 388/761] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:28:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:00.602947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:00.702119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf28c-7aa6-4dcc-b8ac-9c176d853bf2 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.702506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf28c-7aa6-4dcc-b8ac-9c176d853bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.702861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf28c-7aa6-4dcc-b8ac-9c176d853bf2 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.704546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 389/762] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:00.710053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52b439-9764-42f9-8e61-d7db3f91e5e5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.710415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52b439-9764-42f9-8e61-d7db3f91e5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.710744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52b439-9764-42f9-8e61-d7db3f91e5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.745901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9e52b439-9764-42f9-8e61-d7db3f91e5e5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:00.748873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9e52b439-9764-42f9-8e61-d7db3f91e5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:00.768429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47bc60-16a5-4e20-b9e8-9aeb1e1378bd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:00.810736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:00.812864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.814598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.814874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.831932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:00.833335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:00.840452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3562c0ce-29c1-4188-b755-ab2ddb36c5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:00.852381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:00.853294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:00.854145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:00.854145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:00.854145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:00.854145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:00.854145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 374/763] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:00.855496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0d4a-12b4-400e-85e0-beeef462e728 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.856188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0d4a-12b4-400e-85e0-beeef462e728 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.856556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0d4a-12b4-400e-85e0-beeef462e728 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.862065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 375/764] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:00.863040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0d4a-12b4-400e-85e0-beeef462e728 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:00.874777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 390/765] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:00.876525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:00.876955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:00.877394 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:00.896339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:00.898851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:00.934468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:00.996775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0d4a-12b4-400e-85e0-beeef462e728 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.004019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 391/766] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:01.007310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.046802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.048124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.048899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.049968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.051568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:01.052417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.054768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.074174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 376/767] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:01.075355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.136423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ea377-cf64-4bac-be55-382377f8fd72 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.164981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:01.177767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.178791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.179091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.179370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.180405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.181731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:01.197616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:01.199116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:01.200547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:01.203380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 392/768] 10.222.0.74 () {56 vars in 1129 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:01.204261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.239120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:01.243687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a53c-cc09-45c4-87e1-017124346ecc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.285502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.286798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.287378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.287854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.300220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:01.309730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:01.310674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:28:01.311778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:01.315034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 377/769] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:01.316283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.324332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0748415-a4ad-4db9-a438-283e3f356f7e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.362475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.364486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.364973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.366114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.377160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.379641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:01.397866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:01.399079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:01.400556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:01.403962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 393/770] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:01.404895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.448069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc4adc-2087-44eb-b951-6ef95d19e96d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.469399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:01.490815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.491878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.492504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.493098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.494265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:01.494890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.496995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.507731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 378/771] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:01.508987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.563642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:01.593600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ce9-8fc8-4802-b63d-55028362fc75 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.639210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72764-5fd4-4aaa-8add-77203127d5f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.657446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.658456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.658956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.659532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.660965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:01.661514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.664312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.677698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 394/772] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:01.679376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.689371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.690535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.691018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.691536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.692523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:01.693050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.694945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.711403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 379/773] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:01.712636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.765308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:01.842512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd089cc-2f5d-41b5-9501-901d8095836e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.875728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bc12-35ef-4602-a79e-8eb7b07fb052 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:01.884873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.886051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.886548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.887181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.899669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.902370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:01.924644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:01.925562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:01.926421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:01.926421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:01.926421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:01.926421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:01.929303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 380/774] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:01.930507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.937629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:01.938720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:01.939225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:01.939688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:01.940943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:01.941468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:01.944481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:01.960942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 395/775] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:01.962385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:01.986714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.055842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.063810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a09e8-8a2f-4ccf-899e-da539916b21c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.104715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734a6f7-d083-40d8-bba7-deb7dc140abd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.105180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.106516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.107002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.107524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.110357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=b6297dc028a44750a69132257e0aede1, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.112159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.130097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 381/776] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:01 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:02.131895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.183858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.184556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.184842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.185132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.185883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:02.186214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.187683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.214693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 396/777] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:02.220378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.313198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47800d0b-97de-4d78-b76c-070d709351ea None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.347979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.356625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.358065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.359149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.359675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.361499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] RBAC: Adding query filter params (interface=public, service_id=a6a00ec5d79c413aa7edb821efaa33dc, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:02.362212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.364265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525985-1991-455b-b342-e56ae47a3578 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.378246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 382/778] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/endpoints?service_id=a6a00ec5d79c413aa7edb821efaa33dc&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:02.379896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7faefe-9274-48cf-984e-91051202b57c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.380336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7faefe-9274-48cf-984e-91051202b57c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.380929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7faefe-9274-48cf-984e-91051202b57c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.383386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 383/779] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:02.386865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7faefe-9274-48cf-984e-91051202b57c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.388016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508c8f-6ca9-46a7-9a34-d18c92745fc8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.416878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.431674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.432657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.433236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.433618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.442728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.444418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ac2eb108-82d9-486c-b1e6-5ac4856443cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:02.463427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:02.464400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:02.465317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:02.466369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 397/780] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:02.467238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.467791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.471373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.489368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:02.490906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:02.519631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7faefe-9274-48cf-984e-91051202b57c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.571620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.573014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.573705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.574315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.575335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.577545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.585480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 384/781] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:02.591309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.598893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 398/782] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:02.634553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.717019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.746377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa901475-a55d-4633-a0c3-8a80850eff60 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.777900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:02.794078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.794771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.795073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.795352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.796209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:02.796521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:02.798428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:02.823196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 385/783] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:02.828589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.831991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fa4aa-7a97-49cc-99ff-f9829101c722 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:02.873257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:02.874069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:02.874356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:02.874636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:02.886760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:02.894606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:02.897029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:02.899113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 399/784] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:02.905961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:02.985205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc297-6bd4-43ec-af99-456e6a7203d7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.040190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62848304-1ebb-4cd8-99b7-0a00c16b8c54 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.066040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.090287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.090983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.091291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.091570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.092525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:03.092838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.094227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08609ba7-7942-424c-a8c5-573191572121 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.099885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.100971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.100971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.101316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.106245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 400/785] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:03.107690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27359d18-f6b6-4831-9b1d-6ac95cebd810 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.107987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27359d18-f6b6-4831-9b1d-6ac95cebd810 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.108254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27359d18-f6b6-4831-9b1d-6ac95cebd810 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.109764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 401/786] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:03.114501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27359d18-f6b6-4831-9b1d-6ac95cebd810 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.116039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.120619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.122548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0d50caee-15b7-41f3-b577-acf1667725d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:03.147108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:03.149258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 386/787] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:03.151108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.151792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.151792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.172870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:03.174205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:03.262349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27359d18-f6b6-4831-9b1d-6ac95cebd810 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.263315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 387/788] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:03.264391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.305250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.306214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.306570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.306914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.323763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.326230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:03.352438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:03.354477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:03.356870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:03.356870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:03.356870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:03.356870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.356870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 402/789] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:03.359447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.406544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.423094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941cf38-1ac9-48a8-9581-9276ec47e80c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.451344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.466447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.471073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.489058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 388/790] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:03.490325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.501799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdf698-126c-443f-8f9a-ae0b57d4e597 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.545642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.546400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.546876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.547061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.547762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.549168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:03.562639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:03.563904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.564801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 403/791] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:03.565573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.603227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842605a-222a-4855-b214-ca8b38d15347 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.640359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.641041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.641372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.641650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.642448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:03.642757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.644188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.657821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 389/792] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:03.658693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.681652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.686404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f5bd7-57bd-494e-b386-d500822b1de0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.705257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:03.727852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.728664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.728950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.729333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.738606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.740382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:03.757458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:03.759575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:03.761705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.769047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 404/793] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:03.769047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.769506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713711d-63d8-46a7-8c76-db60b4a628ee None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.804807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.805426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.805709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.805987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.806699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] RBAC: Adding query filter params (name=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:03.807007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.808513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.814042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 390/794] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:03.815123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.882724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cd523-dd35-4058-b72d-47da49c04f62 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:03.925606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:03.926434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:03.926720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:03.927001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:03.936217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:03.937717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:03.954648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:03.956942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:28:03.959435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:03.967347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 405/795] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:03.967347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:03.984805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.043084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbad0b4-2fff-48f9-96cc-86e2dda5e288 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.050530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.079407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.079994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.080278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.080555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.081349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:04.081659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.082593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44d430-ad1e-4919-ba49-ec20ac5abc09 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.083132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.100888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 391/796] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:04.100888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.127931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.128996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.129515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.129938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.130861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] RBAC: Adding query filter params (type=network) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:04.131646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.133478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.139421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 406/797] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:04.140465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.201291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67ba1-cbd0-48c1-8822-0183d82c76b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.232680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.235771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.236400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.236684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.236958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.237864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:04.238108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.239645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.256449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d7a56-f20c-40ac-8b06-0ffddd7f43bf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.263120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 392/798] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:04.263120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.298179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.305372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.305372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.305774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.305933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/687bbc1e0d174b78a51d01542d1fd03a/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.317530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=687bbc1e0d174b78a51d01542d1fd03a, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.319947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.343322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 407/799] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:04 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/687bbc1e0d174b78a51d01542d1fd03a/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:04.344287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.384636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96147a-ed55-429d-a643-8358bf361a14 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.427597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.428724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.429207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.429775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.430871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] RBAC: Adding query filter params (interface=public, service_id=425b992cb3234ca78e44e08694517bc0, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:04.431689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.433734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.440735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 393/800] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/endpoints?service_id=425b992cb3234ca78e44e08694517bc0&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:04.446562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.582002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deae26-c25d-4206-83c2-272af6baa814 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.621557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.632002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:04.634428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c4086-5499-483e-8d8c-9ea6286d9f1e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.639569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.640577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.641030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.641442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.657612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.659581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-0d3e8666-b60e-40e1-b9b4-b39b4adbf359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:04.684885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:04.686453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:04.687552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:04.687552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:04.687552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:04.687552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:04.688906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 408/801] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:04.689070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.695421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.695421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.695680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3967c8b-f347-4dfb-b238-4eaa635e9cd4 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.695680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3967c8b-f347-4dfb-b238-4eaa635e9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.695810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.695878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3967c8b-f347-4dfb-b238-4eaa635e9cd4 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.695944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.696021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 409/802] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:04.696102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad209c3-1732-412f-a277-b50f9c3bf7dc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:04.696203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3967c8b-f347-4dfb-b238-4eaa635e9cd4 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.700962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 394/803] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:04.704301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.704588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.704865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.738912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:04.740220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:04.891218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 395/804] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:04.904309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3967c8b-f347-4dfb-b238-4eaa635e9cd4 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:04.921865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:04.976726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:04.977902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:04.978399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:04.978960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:04.980069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:04.980732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:04.984194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.007267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 410/805] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:05.010198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.051115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.080376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc02b8-b80a-4976-8b85-ecfb9e7b2828 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.087783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.136448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.137520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.137786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.138630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.151506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-945efa38-1cd4-4e01-8a70-a8c6bcca7ff5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:05.159621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:05.160771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:05.161598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:05.164334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd970ec9-4367-4f5e-b165-ff1addd9042c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.168082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 396/806] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:05.168977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98f83-c488-4ddf-8cdd-83f51e79af72 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.169331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98f83-c488-4ddf-8cdd-83f51e79af72 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.169675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be98f83-c488-4ddf-8cdd-83f51e79af72 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.171325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 397/807] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:05.172099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be98f83-c488-4ddf-8cdd-83f51e79af72 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.208401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.209983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.210966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.211485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.222568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.225383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-17909b99-ea99-4412-8402-e4fdd2cbf3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:05.246776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:05.247768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:05.248632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:05.251175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 411/808] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:05.256604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.256990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.257290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.281914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:05.283247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:05.308693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be98f83-c488-4ddf-8cdd-83f51e79af72 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.356094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.357166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.357670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.358207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.359263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:05.359792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.360834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 412/809] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:05.361635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.361818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.374607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 398/810] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:05.384246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.394670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.409866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.499701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb25001-d372-4b87-8314-91a9a905f1ae None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.510000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ac100-f018-4cf1-9e0e-aad2af71cde5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.541766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.542474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.543212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.543212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.546862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:05.546862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.546862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.563717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 413/811] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:05.563717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.566359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.567604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.568099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.568597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.581169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.583221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:05.604849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:05.605841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:05.606731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:05.606731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:05.606731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:05.606731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:05.611581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 399/812] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:05.612833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.674778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40970401-06c3-437d-978d-6a8600c33a7b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.675469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.687441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:05.715853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.717022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.717520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.718085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.719021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.720701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.721969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ea331-679b-4d13-a5a5-f6f133a7f40e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:05.734503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:05.735530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:28:05.736356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:05.738998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 414/813] 10.222.0.74 () {56 vars in 1129 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:05.739967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.763821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.765176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.765297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.765641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/3fc2a725e2f14b90be5575a030f47dae/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.768109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=3fc2a725e2f14b90be5575a030f47dae, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.769861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.795351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 400/814] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:05 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/3fc2a725e2f14b90be5575a030f47dae/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:05.796614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.864093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ac743-3003-4ba4-a113-5adfcf94a74c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:05.907591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:05.908598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:05.909349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:05.909779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:05.910732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:05.911870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:05.913836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:05.929617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 415/815] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:05.936408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:05.959575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.059004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be925685-7bf6-4d6b-aef2-03b3d7207c8b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.064233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670a4cc-ae1e-4cef-85a5-6ec1c15a145b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.083370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.110475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.110913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.111479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.111854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.112242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.112444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.112868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.113228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:06.113454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.113712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.122568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.123065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.129033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.132287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 401/816] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:06.138956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:06.146637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:06.147707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:06.148623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:06.152362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 416/817] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:28:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:06.164316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.273498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a000387-52a2-4b0e-b4f9-56b78d5b40f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.281868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.312551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.313379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.313661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.313935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.314627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:06.314931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.316736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.319906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864297b-5f21-448b-b52d-0b428aa6ade4 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.324243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 402/818] 10.222.0.74 () {56 vars in 1124 bytes} [Thu Jun 16 16:28:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:06.329097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.380465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.381708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.382205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.382383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.382688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.383641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:06.384019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.385648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.413930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 417/819] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:28:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:06.429682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.456583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9f1ed-4747-4a4e-8bba-bf152e99c19d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.492301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.493274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.493759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.494202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.495143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.497341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc66b4-430a-4039-959c-ee256391b41c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.522496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 403/820] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:06.572414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d038d8-650b-4923-b74b-834941719179 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:06.613930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.618357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:06.619008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.623705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.624001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.718346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:06.719916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:06.773937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948538-e201-41da-9a68-5a74ffce5360 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.774283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948538-e201-41da-9a68-5a74ffce5360 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.774508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948538-e201-41da-9a68-5a74ffce5360 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.776146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 404/821] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:06.782248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:06.782534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:06.782814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:06.802142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:06.803507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:06.857851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 418/822] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:06.864254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.903335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 405/823] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:06.908993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.929742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:06.965657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:06.997981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083d0a-ada5-4bc4-804b-9a838e61f3dd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.036828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.037926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.038236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.038512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.050773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.052449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:07.072558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:07.073495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:07.074591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:07.074591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:07.074591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:07.074591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:07.074591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 419/824] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:07.076030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8110bf5-9934-4a3f-95fa-e3a7c2ad0a27 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.079779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.130328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.131389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.131861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.132142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.159017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:07.174636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:07.175805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:28:07.176816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:07.180266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 406/825] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:28:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:07.186084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.224429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:07.258505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e7ab8-36b6-48b2-9401-851e1355368d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.329266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.330712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.330999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.331299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.332044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:07.332355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.333758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.356906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 420/826] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:07.362397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.380881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:07.405256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a01bd-8ac0-484f-adda-f2cbb9f7d06a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.476431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d345a1-c997-4494-be8f-9bb662f817df None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.499150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.499843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.500132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.500418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.501174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:07.501653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.502875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0391ac-2c3f-400a-8346-f54d9ef07715 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.512660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.513716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.514254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.514684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.520480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 407/827] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:07.522060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5190fe-534c-412d-92e1-8945c9126382 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.522603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5190fe-534c-412d-92e1-8945c9126382 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.522976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5190fe-534c-412d-92e1-8945c9126382 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.523288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.524668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 408/828] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:07.525255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.529432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5190fe-534c-412d-92e1-8945c9126382 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-5a90ed13-3fc1-4b67-bebe-57844a9ae18c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:07.541007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:07.542106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:07.543219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:07.545391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 421/829] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:07.551313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.552442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.553218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.559865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:07.573331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:07.574644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:07.671480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5190fe-534c-412d-92e1-8945c9126382 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.710690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.711556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.711844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.712125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.725978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.727519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:07.746772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:07.747752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:07.748828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:07.748828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:07.748828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:07.748828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:07.748828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 409/830] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:07.749637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.753183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:07.775844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 422/831] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:07.777364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.852622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72261-2de7-466e-b540-6429c1078da4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.887088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4369168-f7a7-4e87-a89f-49643f4797b6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:07.890704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.891255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.891612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.891965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.892838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:07.893219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.894735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.913823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 410/832] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:07.913823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.934408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:07.935978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:07.936763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:07.937344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:07.938368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:07.938878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:07.940601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:07.942610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:07.956319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 423/833] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:07.957697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:07.978364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.017151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae3ab6-fee2-4042-967c-239aafda45a7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.054766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.055607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.055894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.056165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] PATH_INFO: `/v3/services/network` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.056826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.058212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:08.070405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:08.071427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:08.072264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 411/834] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:08.073069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.073810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eac44-9157-4906-b7df-4dde81c35c16 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.112406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.113362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.113788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.114204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.115410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:08.115889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.117647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.131268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 424/835] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:07 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:08.132625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.181541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c4dc1-5be0-4841-8c1c-c7f3438dfffd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.211644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.224996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891431-7646-423c-b845-854c59285b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.225874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891431-7646-423c-b845-854c59285b1a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.226193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891431-7646-423c-b845-854c59285b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.226472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891431-7646-423c-b845-854c59285b1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.239269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e891431-7646-423c-b845-854c59285b1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.240819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e891431-7646-423c-b845-854c59285b1a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.271521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1e891431-7646-423c-b845-854c59285b1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:08.273403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:08.274339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:08.275303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:08.278867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 412/836] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:08.284134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891431-7646-423c-b845-854c59285b1a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.352975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46aafe-2599-4110-85c8-b53f03273246 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.401648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.402331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.402614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.402888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.403631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] RBAC: Adding query filter params (name=network) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:08.403942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.408704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bead6e0-acac-4130-a86b-608c4ffa369a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.417086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 425/837] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:08.420030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f1bc-5e02-4cde-9087-a77ee2d76964 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.420572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f1bc-5e02-4cde-9087-a77ee2d76964 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.421074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f1bc-5e02-4cde-9087-a77ee2d76964 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.422660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 426/838] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:08.423752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7f1bc-5e02-4cde-9087-a77ee2d76964 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.454697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891431-7646-423c-b845-854c59285b1a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.502623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.503300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.503589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.503872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.505012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:08.505333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.507870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a4a-0026-4c52-8e25-5015d35242ea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.533810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 413/839] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:08.533810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.533810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.533810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.558790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:08.560383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:08.562959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7f1bc-5e02-4cde-9087-a77ee2d76964 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.590727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.604627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.605737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.606222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.606692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.607658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] RBAC: Adding query filter params (type=network) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:08.608181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.609857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.620902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.621589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 427/840] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:08.623908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.655087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 414/841] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:08.655087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.769144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb453-7aee-490c-b01f-73cd97827c7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.769874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e38d-6bee-4620-a4dd-8002126c2c5c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:08.811282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.811957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.812249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.812528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.813280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:08.814874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:08.815555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:08.816765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:08.817241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:08.817723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:08.843126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 415/842] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:08.843126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:08.881446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.966192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:08.982184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d628bae-71c3-4e3f-a8ad-2852dd10d5cf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.004594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.006158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.061859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.062604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.062890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.063216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.086958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:09.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.096016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:09.096836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:09.099024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 416/843] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.107545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.182086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 428/844] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:09.188147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.234499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.252869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3052-bef2-4919-8d11-d0ffdaafd0e5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.283491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.294732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.295457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.295741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.296202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.296750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:09.297055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.298429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfaaf0a-6fae-4e25-9e19-dd86bf7e9fe7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.299084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13aaa9-fb68-4a8c-94f0-25d979069f61 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.311696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 417/845] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:09.311696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83c8df-ca63-4689-bee1-84c1e0bb416e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.311696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83c8df-ca63-4689-bee1-84c1e0bb416e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.311696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83c8df-ca63-4689-bee1-84c1e0bb416e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.311696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 418/846] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:09.312714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83c8df-ca63-4689-bee1-84c1e0bb416e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.343134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.343945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.344228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.344507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.357194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.359033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-44f3a949-0628-476f-a20c-d944a31338af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:09.378633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.379808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.380662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:09.380662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:09.380662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:09.380662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:09.380662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 429/847] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.381669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.381855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.382104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.400519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:09.401734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:09.428960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83c8df-ca63-4689-bee1-84c1e0bb416e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.464490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.465299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.465582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.465854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.476982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.477589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 430/848] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:09.478881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.479020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:09.498298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.500369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:09.500369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:09.500369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:09.500369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:09.500369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 419/849] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.505528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.508544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.563189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.598442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa187888-0b8b-4a52-8666-549e33c544b0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.619875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940a32b-af4a-4e38-b62d-8682059d2221 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.634063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.634734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.635022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.635334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.636063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:09.636374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.637744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.653664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 431/850] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:09.653664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.660961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.661632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.661915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.662245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.663102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:09.663416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.664963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.682507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 420/851] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:09.682507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.745036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.757178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603bd5c-1cad-4b3a-8739-3aabfc16c1a5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.783806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:09.789524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54ca56-cd58-49e2-9cba-8c78aaa6e1b8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.793333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.794150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.794435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.794710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.795396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.796773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.810283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.811318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:09.812112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 432/852] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.813464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.830782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.831622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.831915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.832167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.840513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.841962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:09.856121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.857219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:09.858646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 421/853] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.862569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:09.926825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396809-13c3-44b4-ad8f-ae1550c4c997 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.964313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:09.965129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:09.965413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:09.965688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:09.972679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da258331-97d0-436e-a64b-e8ee85d0dda4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:09.973976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:09.975476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:09.990305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:09.991338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:28:09.992467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:09.996694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 433/854] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:09.997564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.000467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.013714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.014389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.014676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.014953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.015778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:10.016094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.017621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.022818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 422/855] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:10.023554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.059175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.109046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5684843-c5d2-4a07-bb78-8839edd41ad5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.131867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68343fc0-e843-48fa-9ecc-2e4dfae67a90 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.145969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.146893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.147285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.147567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.148361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:10.148676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.150106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.167733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 434/856] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:10.167733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.171757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.172450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.172741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.173000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.173849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:10.174158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.176213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.198494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 423/857] 10.222.0.74 () {56 vars in 1135 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:10.199916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.249539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.280161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16759697-94b9-460b-ae87-dd16a19ecb68 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.296036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.307431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1223-2b62-4a58-8278-4ce352bc84bf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.321326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.322101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.322397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.322668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.323401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:10.323704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.325283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.339082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 435/858] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:10.339082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.353802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.354607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.354960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.355350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.356752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=687bbc1e0d174b78a51d01542d1fd03a) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:10.357253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.358827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a30005-156f-475d-a592-e2b10fcd82ae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.366499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 424/859] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/role_assignments?user.id=687bbc1e0d174b78a51d01542d1fd03a&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:10.463225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5feea-230a-4e53-85bd-a2c0429345a3 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.499746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ea9b1-1bb4-4217-abdd-555ab1b3c4fe None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.500212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ea9b1-1bb4-4217-abdd-555ab1b3c4fe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.500593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ea9b1-1bb4-4217-abdd-555ab1b3c4fe None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.502426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 425/860] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:10.508495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.509170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.509679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.512316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.513020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.513325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.513608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.530115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:10.535411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:10.566438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.594610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.596104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.633711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 426/861] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:10.658094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.713356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.725598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 436/862] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:10.731517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.795982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3082-2986-4806-a0bc-d116ddf2184b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.825421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:10.841242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.842071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.842355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.842632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.843313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.845196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.858845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dade6-106e-460b-abe7-e38778db8aa9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:10.863158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:10.864311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:10.866019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 427/863] 10.222.0.74 () {56 vars in 1132 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:10.873789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.910555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:10.911327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:10.911610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:10.911884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:10.923260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:10.924712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:10.944672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:10.946024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:10.946891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:10.946891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:10.946891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:10.946891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:10.946891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 437/864] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:10.952419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:10.995087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819fece-34e5-44d3-8090-440d81d7536b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.003416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:11.033144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.033827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.034115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.034396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.035186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:11.035606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.037232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.043000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 428/865] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:11.048381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:11.061179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae026d1-5229-478e-bf9f-51570bc14d7e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.098495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:11.109388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.110652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.111011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.111395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.112228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:11.112608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.114415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.131487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 438/866] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:11.136869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:11.179999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48261e01-35bf-487e-8c88-94f185eeb7e2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.215505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.216188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.216474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.216752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.217700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:11.217809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.219174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.231078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 429/867] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:11.252777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96b7ab-7583-47ec-93b7-e1ab6be1d52d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.290614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.291503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.291842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.292118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.300090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.301687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:11.317796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:11.318935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:11.319998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:11.323192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 439/868] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:11.325546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:11.373995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:11.473104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b363-f50d-45e5-8620-91d079bfc400 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.520493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.521176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.521521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.521806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.522618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:11.522938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.524465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fbc08-b203-47e3-b0e5-0657d3a02d1b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.540234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 430/869] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:11.546022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:11.612848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:11.689600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73652-b0b7-4bc6-8791-bd4dd04d1b63 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:11.757735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=3fc2a725e2f14b90be5575a030f47dae) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:11.763145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40744-ab18-49e1-9e82-2ba86be7fad6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:11.783087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 440/870] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:11 2022] GET /identity/v3/role_assignments?user.id=3fc2a725e2f14b90be5575a030f47dae&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:12.672645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103f2f5-71dd-47e1-9494-9ef3ce538e6c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:12.673211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103f2f5-71dd-47e1-9494-9ef3ce538e6c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:12.673211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103f2f5-71dd-47e1-9494-9ef3ce538e6c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:12.674948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 431/871] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:12.680972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:12.681526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:12.681526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:12.700373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:12.701621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:12.739523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9920-6a35-4334-b83d-5f40a89aa844 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:12.739846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9920-6a35-4334-b83d-5f40a89aa844 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:12.740081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b9920-6a35-4334-b83d-5f40a89aa844 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:12.741789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 432/872] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:12.748495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:12.749140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:12.749618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:12.767310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:12.768985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:12.776074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 441/873] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:12.798873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:12.852274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:12.859078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 433/874] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:12.881131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:12.907772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:12.919581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64188288-a9e2-4541-b223-dedb2822af19 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:12.956737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:12.957820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:12.958315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:12.958816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:12.960675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:12.962761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:12.987539 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 442/875] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:12 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:13.019644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0edb33-68e0-4eab-8975-b5c70c9be8fc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.065593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.066392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.066678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.066957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.079262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:13.088308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:13.089311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:13.090112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 434/876] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:13.098162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.218813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408bbff-0d53-49fb-8b2e-3aa5d66b658c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.220947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:13.259724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.260475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.260778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.261108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.261992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:13.262302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:13.263869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:13.273665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 443/877] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:13.279584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.363979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:13.389976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701ab45-b3ce-473c-82fe-16464a132d77 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.427437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.428863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.429203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.429481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.441888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:13.443604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fbb14558-443a-4ec4-b7ee-77bd0fa26360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:13.462156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:13.463079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:13.463951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:13.463951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:13.463951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:13.463951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:13.463951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 435/878] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:13.469586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.597167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df103893-cd2a-45ce-9b77-6324a9c4aeb7 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.610660 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4e17-4948-4bf9-b1ae-5f2c3902821a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.610968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4e17-4948-4bf9-b1ae-5f2c3902821a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.611408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4e17-4948-4bf9-b1ae-5f2c3902821a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.613183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 436/879] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:13.618570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.618822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.619078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.636678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.637762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.638245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.638717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.639664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:13.640053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:13.640406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:13.641328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:13.642187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:13.655894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 444/880] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:13.660875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.692446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:13.727274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 437/881] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:13.746737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.793684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11530916-429a-4eb9-8de4-1975d684cf61 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.827437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:13.840332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.842262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.842777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.843321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.853927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:13.856506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:13.871468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090de4f-a9f6-42b2-9ebf-d25679beb29b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:13.876982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:13.877895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:13.878810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:13.886387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 445/882] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:28:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:13.888568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:13.914249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:13.915222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:13.915506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:13.915780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:13.916952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:13.918360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6a7d-488f-455f-a41b-f9247ff004d5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:13.947087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 438/883] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:13 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:14.006576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.028574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9388f-3717-4723-b6a0-1d2d980ea115 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.076372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.077057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.077334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.077603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.078537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:14.078844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.080114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2868a1-3e68-48c2-af25-2e08c40345d0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.080324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.080421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2868a1-3e68-48c2-af25-2e08c40345d0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.080678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2868a1-3e68-48c2-af25-2e08c40345d0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.086375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 439/884] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:14.088461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.088752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.104336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 446/885] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:28:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:14.106530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:14.111117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:14.112341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.207207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 440/886] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:14.223998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.247367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85040030-0857-43da-b2ab-5158a5902710 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.291808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.294309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.295370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.295718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.296062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.309569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.372429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c59f1-c512-4581-b5be-9a6614e056bb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.377729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.379280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.412381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.413392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.413735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.414075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.414948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.416279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:14.428870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:14.429914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:14.430738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 441/887] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:14.435939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.513824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 447/888] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:14.514712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.570780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.574405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d345a-0b2e-4313-85f3-6b33afd4e0fd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.589007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.612958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.613917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.614349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.614777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.616122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:14.616592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.618140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.624093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 442/889] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:14.630392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.658699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9be5a-b7b3-4a0c-a635-bf557b8fa56d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.709249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.710205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.710489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.710762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.723981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.725559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.743312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a02bc2-0e1a-4495-850c-dc10ac51d00f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:14.748503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:14.749353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:14.750235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:14.750235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:14.750235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:14.750235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:14.750235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 448/890] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:14.759352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:14.793351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.794037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.794385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.794725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.795783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:14.796184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.798254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.807146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 443/891] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:14.873001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:14.885392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d22b1d-939c-4a1b-bedf-3a0b0368e3e5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:14.925897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:14.926849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:14.927386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:14.927842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:14.929061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:14.929569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:14.931199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:14.949252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 449/892] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:14.955257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.006245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:15.072730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9d4ea-d030-4f16-9360-2da32c956ccd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.109649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.110479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.110811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.111165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.125118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.126395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-76ec06da-da4e-4109-b335-3a4fde760bb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:15.143965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:15.145059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:15.145983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:15.148907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 444/893] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:15.154887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.265900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf3c16-2b24-4911-9f6b-128bc9b1c652 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.300404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.301106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.301390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.301661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.302447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:15.302842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.304482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.307591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1d99-05c9-41b7-b180-12058e1ddef2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.307873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1d99-05c9-41b7-b180-12058e1ddef2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.308150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1d99-05c9-41b7-b180-12058e1ddef2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.309748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 445/894] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:15.315030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.315344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.315620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.319334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 450/895] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:15.326615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.335100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:15.335100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:15.339707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:15.431092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 446/896] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:15.448139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.452269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbb81-e9e5-434e-929e-4624b909948a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.478880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:15.494502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.496351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.496868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.497741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.499044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] RBAC: Adding query filter params (role.id=02a72f7b4ffa4327bc7c7db26907444d, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=687bbc1e0d174b78a51d01542d1fd03a) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:15.499807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.502276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.511179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 451/897] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/role_assignments?user.id=687bbc1e0d174b78a51d01542d1fd03a&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=02a72f7b4ffa4327bc7c7db26907444d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:15.563632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8e5a-cd79-4213-93d1-ba69252c8519 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.603116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.603915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.604200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.604478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.605205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.606613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:15.619458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:15.620958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:15.621839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 447/898] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:15.627221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.713342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:15.741613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362518c9-13c5-46f7-9e75-d8d08f3fcea6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.777581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.778261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.778540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.778814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.779545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:15.779850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.781221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.786588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 452/899] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:15.792346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:15.890609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:15.899108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b83ad-3409-4ad2-a6d2-59fe5f58f736 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:15.937904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:15.938579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:15.938860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:15.939160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:15.939866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:15.940209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:15.941611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:15.947119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 448/900] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:15.952556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.060506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1999e4-4856-4177-b379-e3202353192e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:16.096150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:16.096814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.097098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.097369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.098163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] RBAC: Adding query filter params (interface=public, service_id=765a434e02d3434ea93bf9dbc55cbb6d, region_id=RegionOne) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:16.098473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:16.100007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00ce97-57a3-4811-93e5-788fbf8650f9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:16.108395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 453/901] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:15 2022] GET /identity/v3/endpoints?service_id=765a434e02d3434ea93bf9dbc55cbb6d&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:16.116292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.226220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a85e9-efc6-4a45-ae8e-fe9554923793 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:16.262943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:16.263649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.263930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.264203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.264882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:16.266254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145f1ba-ffe8-4079-9d5a-48ca6246b956 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:16.272597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 449/902] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:16 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:28:16.281240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80d62-3903-4871-a1e8-954c6bfa5b95 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.281566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80d62-3903-4871-a1e8-954c6bfa5b95 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.281806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80d62-3903-4871-a1e8-954c6bfa5b95 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.283398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 454/903] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:16.289453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.289806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.290008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.308307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:16.309540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:16.328827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:16.391183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 450/904] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:16.408879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda80d62-3903-4871-a1e8-954c6bfa5b95 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.411926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:16.520822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda80d62-3903-4871-a1e8-954c6bfa5b95 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:16.556509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:16.557332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.557616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.557892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.558551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:16.559956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:16.572557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:16.573728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:16.574726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:16.578740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 455/905] 10.222.0.74 () {56 vars in 1132 bytes} [Thu Jun 16 16:28:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:16.580163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.686804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b0b4-381a-47b1-a63f-d4230d9d9928 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:16.722460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:16.723161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.723447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.723719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.724429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:16.724826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:16.726444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:16.735168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 451/906] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:16.737519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.807515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:16.846444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f1d6-dac5-4492-8feb-52ea749a46f8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:16.882622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:16.883643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:16.884109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:16.884583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:16.885486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:16.885988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:16.887577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc9f9-5512-418a-8fe6-875eac757f39 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:16.893243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 456/907] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:16.903082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:16.919482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:17.017940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68f34d-0ff0-4243-9551-3f55c1c296fe None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:17.054868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:17.055576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.055861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.056134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.056922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] RBAC: Adding query filter params (interface=public, service_id=7f0748c9eed440cd8b74c5acceb08e31, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:17.057234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:17.058663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:17.064681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 452/908] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:16 2022] GET /identity/v3/endpoints?service_id=7f0748c9eed440cd8b74c5acceb08e31&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:17.066298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dab-1ea1-4247-a1b7-f9d4614feb13 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.066588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dab-1ea1-4247-a1b7-f9d4614feb13 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.066857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dab-1ea1-4247-a1b7-f9d4614feb13 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.068585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 457/909] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:17.070403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:17.074609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.074894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.075197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.092931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:17.094170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:17.175457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 458/910] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:17.175788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:17.192505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95eac28-19a6-4695-be9b-9a154717a023 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:17.197042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:17.238714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:17.239437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.239727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.240007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.240729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:17.242123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52b3fd-f286-4c41-bae0-650d4e09bc37 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:17.251127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 453/911] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:17 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:28:17.273733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:17.317637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589db9c-d79b-4478-8cef-576ce198115c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:17.356242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:17.356922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.357201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.357470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.358195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:17.359696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0c522-0d99-4152-a73b-1a3afdfd83f0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:17.377843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 459/912] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:17 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:17.707498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110c380-f214-4ac6-a46a-b10729ce8f18 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.707891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110c380-f214-4ac6-a46a-b10729ce8f18 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.708094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110c380-f214-4ac6-a46a-b10729ce8f18 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.709668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 454/913] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:17.715237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.715532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.715811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.733014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:17.734217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:17.812668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 460/914] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:17.835687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110c380-f214-4ac6-a46a-b10729ce8f18 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:17.946085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110c380-f214-4ac6-a46a-b10729ce8f18 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:17.983790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:17.984853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:17.985281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:17.985698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:17.997220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:18.005671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:18.006810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:18.008159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:18.010531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 455/915] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:18.016706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.066901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.126717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65912f3-5283-481c-a44b-4cfe86932fc9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.163710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.164389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.164675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.164948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.165698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:18.166006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.167407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.176891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 461/916] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:18.184141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.189412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.295438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5303-1bcb-4a98-8182-4a2e2e0e535e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.330500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.331636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.332099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.332580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.344678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.346381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c0ea3153-ad5a-4266-894f-a06eedd23348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:18.364109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:18.365036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:18.366059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:18.366059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:18.366059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:18.366059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:18.368130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 456/917] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:18.373863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.487922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48987e-03f0-414b-994a-8b863571db97 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.498084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.521156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbdb1-8ca6-456d-95a2-a1fde5c9be89 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.521627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbdb1-8ca6-456d-95a2-a1fde5c9be89 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.521733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbdb1-8ca6-456d-95a2-a1fde5c9be89 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.523570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 457/918] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:18.524873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.525607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.525895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.526173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.526910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:18.527548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.528641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.528928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.529168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.529272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.543601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 462/919] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:18.550987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.557149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:18.558411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:18.625472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.648268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 458/920] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:18.667082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d6d27-575c-4b7e-8b15-2a6394728961 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.670905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.702616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.703487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.703770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.704048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.712093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.713559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:18.729259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:18.730255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:18.731250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 463/921] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:18.736258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.765246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.776267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d4f2-3c28-4d6a-896a-5334c0cb3a75 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.819172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.819723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.820071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.820415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.821180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.823736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:18.844981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:18.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:28:18.846803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:18.847937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 459/922] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:18.849228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d287c-e054-4738-b438-ea3e368b53df None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:18.855563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.900537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:18.901578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:18.902066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:18.902605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:18.903840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:18.904388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:18.906139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:18.914094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:18.929496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 464/923] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:18.937778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:18.967206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016a58-df3f-45ce-9b4d-bb140ce2bfc3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.005084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.006281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.006789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.007336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.008668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:19.009241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.012061 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.021151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 460/924] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:19.027684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.032194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.049363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ecdad-df71-4c25-853e-61ded165aa3b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.085798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.086612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.086894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.087202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.097955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.099443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:19.117256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:19.118246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:19.119130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:19.119130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:19.119130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:19.119130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:19.123482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 465/925] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:19.124446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.148330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388bd93-b7fb-4e3f-811c-dca098034f6f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.164616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.189385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.190462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.190943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.191556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.192470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:19.193054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.194976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.201207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 461/926] 10.222.0.74 () {56 vars in 1114 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:19.207188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.227657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf38fe1-80ad-4843-a4db-2f2b87970350 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.267327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.268457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.268751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.269029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.269760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:19.270071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.271588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.280278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.287127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 466/927] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:19.291875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.338998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c826e17-d2b0-40e4-8ef6-12a8ae061cd6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.378880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.379734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.380083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.380422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.381518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.382883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da041a92-cc89-44a6-a987-341c11d9c41c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.406843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 462/928] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:19.411156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50612d1b-5315-4778-843b-bc6333e6734d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.460069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.460587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.460871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.461146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.470512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.473393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.486869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143ed9-0471-4998-8ecf-d76d7740a37f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.487224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143ed9-0471-4998-8ecf-d76d7740a37f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.487457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143ed9-0471-4998-8ecf-d76d7740a37f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.489173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 463/929] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:19.495419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.495712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.496326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:19.496543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:19.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:19.499102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 467/930] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:19.503822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.518873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:19.520192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:19.560898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.587205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.620030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 464/931] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:19.641234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6d8d-5c0d-41a3-b5a1-afaf3adfd7b8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.650133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.689043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.690056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.690514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.691006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.691831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:19.692107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.693602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.706684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 468/932] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:19.713404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.777026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e9465-2418-4067-8d22-ad3a7a9a8099 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.817365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.818016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeca93-1aaf-4284-82e6-214d48a6081c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:19.818195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.818511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.818785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.819476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.820937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.832326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-490c6fdc-d5e3-4717-98e3-b2a826aea598 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:19.833503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:19.840266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:19.841121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 465/933] 10.222.0.74 () {56 vars in 1132 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:19.841937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22815466-e7cb-47e8-bd5a-ac9247d37d5c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.841937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22815466-e7cb-47e8-bd5a-ac9247d37d5c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.841937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22815466-e7cb-47e8-bd5a-ac9247d37d5c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.842898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 466/934] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:19.843751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22815466-e7cb-47e8-bd5a-ac9247d37d5c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:19.856925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:19.858087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.858367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.858767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/687bbc1e0d174b78a51d01542d1fd03a/roles/02a72f7b4ffa4327bc7c7db26907444d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.861501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=687bbc1e0d174b78a51d01542d1fd03a, role_id=02a72f7b4ffa4327bc7c7db26907444d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:19.863421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2ee4-d92a-4694-a068-1a63a98f3cc3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:19.873720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:19.881565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 469/935] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:19 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/687bbc1e0d174b78a51d01542d1fd03a/roles/02a72f7b4ffa4327bc7c7db26907444d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:19.883133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:19.883662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:19.884133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:19.902910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:19.904665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:19.989565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22815466-e7cb-47e8-bd5a-ac9247d37d5c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.000199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 470/936] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:20.023624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.044000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.044906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.045193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.045874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.046629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:20.046945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.048643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.062720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 467/937] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:20.063751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.150864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d23d5-fa2c-4bd5-a8cf-7798f7324e37 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.154843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:20.178886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:20.189557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986314-6239-4690-87d9-e6f5d3b184f7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.202909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.204118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.204910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.205254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.206240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.208320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:20.224536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:20.225623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:20.226427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:20.227184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.227184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.227184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.227184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.228032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:20.228032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.229448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.230784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 471/938] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:20.233104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.235768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 468/939] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:20.241284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.356465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3452a-491c-4488-8cea-e1ce8bd20f2f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.377704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae5907-f5e0-4c75-925f-d6657bd5de5c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.395286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.396062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.396352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.396682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.397709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.399126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.419882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.420478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.420758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.421033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.421791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:20.422100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.423619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.427588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 469/940] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:20.434291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 472/941] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:20.435873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.454088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:20.558328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a855a17-03e5-4429-aa70-a1520b481c43 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.596358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.597040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.597324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.597603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.598351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:20.598667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.600154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.605820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 470/942] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:20.611642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.649242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:20.723797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469386fe-ce59-4ab7-b386-577eed5fc321 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.760650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.761381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.761745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.761942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.762742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] RBAC: Adding query filter params (interface=public, service_id=cec58384051446688c1433cf86fce1d4, region_id=RegionOne) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:20.763078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.764469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ba32d-d2e6-4745-994e-648c94db7114 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.770407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 473/943] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/endpoints?service_id=cec58384051446688c1433cf86fce1d4&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:20.776423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:20.850398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:20.881994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227207cb-94b7-452a-bc55-81674ebaba77 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:20.916491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:20.917164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:20.917450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:20.917727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:20.918413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:20.919811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268dfb0-2f9b-4d79-9462-23ac05f45205 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:20.926317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 471/944] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:20 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:28:21.740004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c097cd3-d2b3-4afd-880c-c20342f774f6 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:21.740559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c097cd3-d2b3-4afd-880c-c20342f774f6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:21.741273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c097cd3-d2b3-4afd-880c-c20342f774f6 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:21.742950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 474/945] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:21.748573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:21.749022 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:21.749196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:21.766672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:21.767926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:21.842512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 472/946] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:21.866671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c097cd3-d2b3-4afd-880c-c20342f774f6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:21.966670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:21.973646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c097cd3-d2b3-4afd-880c-c20342f774f6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.010061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.010721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.011000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.011299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.012201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:22.013635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:22.014999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING keystone.server.flask.common [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 16:28:22.036511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.sql.core [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:28:22.036511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 16:28:22.036511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [parameters: {'id': '497ee9b62d7845a892d852d7bdb0bff5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 16:28:22.036511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80340) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9a6b7f36-e78b-4dea-b5d4-47895ff4f5a0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 497ee9b62d7845a892d852d7bdb0bff5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 497ee9b62d7845a892d852d7bdb0bff5. Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:28:22.045232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 16:28:22.046135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application yield resource Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise self.value Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application session.commit() Jun 16 16:28:22.047098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.flush() Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application compat.raise_( Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 16:28:22.048000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 16:28:22.048896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:28:22.049663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [parameters: {'id': '497ee9b62d7845a892d852d7bdb0bff5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.050507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:22.053580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 497ee9b62d7845a892d852d7bdb0bff5. Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.060222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 475/947] 10.222.0.74 () {60 vars in 1140 bytes} [Thu Jun 16 16:28:21 2022] POST /identity/v3/projects => generated 262 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:22.066227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.134919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e6c7b-00b6-48ea-9064-98317a38ad96 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.135235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e6c7b-00b6-48ea-9064-98317a38ad96 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.135419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e6c7b-00b6-48ea-9064-98317a38ad96 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.139589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:22.141764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 476/948] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:22.149208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.149498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.149778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.173070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:22.175186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:22.182080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5843fa-917c-42a2-bc85-b9bccec6bab2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.216942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.217752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.218035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.218312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.225429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:22.226892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:22.241466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:22.242434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:22.244292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 473/949] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:22.254686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.259587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 477/950] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:22.284322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.355615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f844-53f7-4375-a380-cc2a1810f620 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.375198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:22.383545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:22.391741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.392530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.392818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.393096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.393981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:22.394326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:22.396161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:22.398349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ec685-f79a-400a-a480-0e900912bf00 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.415173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 474/951] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:22.445997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.446851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.447163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.447437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.460945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:22.469785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:22.470760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.472217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 478/952] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:22.478021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.593733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9869b-3db8-427c-ae83-b4a7281e729a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.633727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.634417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.634701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.634981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.635774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:22.636077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:22.637507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0abffa-a46e-4758-a9f4-e3582e205ae7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:22.646808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 475/953] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:22.653452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.756306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9785e1-02eb-4d4e-b280-b549c0c16463 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.757048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9785e1-02eb-4d4e-b280-b549c0c16463 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.757048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9785e1-02eb-4d4e-b280-b549c0c16463 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.759640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 476/954] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:22.763707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.764018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.764294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.767433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457123f-c7c8-43da-a61c-13de3532da44 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:22.785570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:22.786907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:22.815305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:22.816554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:22.816995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:22.817387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:22.831627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:22.833725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:22.835032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:22.836812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:22.859798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:22.860981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:22.862221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:22.862221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:22.862221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:22.862221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:22.862221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 479/955] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:22.868981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.875196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 477/956] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:22.893608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:22.983435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6414bf-e72b-44cf-84a2-43831a3d2e63 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.001409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa4fd-a4d3-48db-981f-6dc37ef6b14d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.020958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.021921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.022614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.022896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.023668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:23.023981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.025498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.040119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 480/957] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:23.051205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.052065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.052170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.052350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.052599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.053354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.054922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:23.071147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:23.072320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.073171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 478/958] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:23.078666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.134075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.136855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.184618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475193d-35d8-4e27-8ca1-4cca61222151 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.199492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d06e5-6ef7-4b02-a05a-10ace2b14038 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.220886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.221709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.221991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.222262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.229751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.231228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.240723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.241617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.242042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.242466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.243346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:23.243810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.245312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d62f26db-6419-427f-ab71-5ee8bf54682f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:23.246602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:23.247714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:28:23.248821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:23.252458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 481/959] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:23.253777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015224d8-4b73-4c6d-b08d-64dc93e3a2c1 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.254067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015224d8-4b73-4c6d-b08d-64dc93e3a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.254353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015224d8-4b73-4c6d-b08d-64dc93e3a2c1 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.259367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 482/960] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:23.259367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015224d8-4b73-4c6d-b08d-64dc93e3a2c1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.259590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 479/961] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:23.260991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.370495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9637d3-072a-4c97-9726-b19beaab4f6f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.371768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015224d8-4b73-4c6d-b08d-64dc93e3a2c1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.380160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.387422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.409183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.409861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.410199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.410421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.411294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:23.411607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.413203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.413817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.414493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.414891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.415113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.415772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:23.416084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.417503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbed5a9-7ce1-41d2-a4a7-42c87dc76f9b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.431473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 480/962] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:23.431473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.431473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.431473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.437440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 483/963] 10.222.0.74 () {56 vars in 1149 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:23.443013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.456768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:23.458682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:23.567498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 481/964] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:23.582847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e085d3-f25c-46c2-9c14-9853b026a53d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.584910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.630638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.631423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.631706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.631982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.676628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.704999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.706472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.712088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.715693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a52814-9d38-42c1-a8ea-b2b4672c2676 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.761624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.762473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.763199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.763482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.764435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.766003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:23.782671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:23.783808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:23.784653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 482/965] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:23.791560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.818527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 484/966] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:23.823781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.910545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:23.922485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a59-6997-4641-9698-8a8caf4f926c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.944420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de79f7-ebc5-40cf-88b9-7a2a773f0eab None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:23.965875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.967140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.967530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.967906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:23.968724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:23.969136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:23.971172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:23.983128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 483/967] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:23.985754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:23.990491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:23.991376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:23.991725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:23.992022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.003682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.005224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.021127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:24.029660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:24.030553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:24.031476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:24.031476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:24.031476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:24.031476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:24.032930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 485/968] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:24.038131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.101136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0f04d-aab0-416c-ad93-6ef93607371f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.136260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.137106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.137407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.137682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.138854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:24.139212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.140893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.151213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 484/969] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:24.155668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.159210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded4da-5ad9-4795-8151-5044eae4844a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.185254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:24.198788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.200209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.200714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.201245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.202618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:24.203367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.204945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.222179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 486/970] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:24.227676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.277938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:24.294940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c71d3-3951-41b8-a5c9-2feaf6466cdb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.343406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.344423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.345120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.345562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.347172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.348824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.349090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc8ba-8e9f-4af8-ab1a-1b0a2695b699 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.383509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 485/971] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:24.405149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.405964 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.406250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.406580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.414849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.416382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:24.432305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:24.433298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:24.434180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 487/972] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:24.439884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.563372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf63e7b-56ed-4ff7-ae93-faa5c4fb8939 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.604003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.604681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.604964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.605295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.606041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:24.606350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.607828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f2be0-7df4-4406-9f13-a02e75669833 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.621998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 486/973] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:24.628434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.675567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:24.743937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74954-e8f7-4473-8291-3a171c5b5fbd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.746752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cead15-4ec6-4241-a915-0164a9049c1b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.747041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cead15-4ec6-4241-a915-0164a9049c1b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.748100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cead15-4ec6-4241-a915-0164a9049c1b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.749956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 487/974] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:24.753886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.754174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.754451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.772216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:24.773611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:24.775926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:24.793871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:24.795039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:24.795557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:24.796376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:24.797432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] RBAC: Adding query filter params (role.id=02a72f7b4ffa4327bc7c7db26907444d, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=687bbc1e0d174b78a51d01542d1fd03a) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:24.797929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:24.799609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:24.808992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 488/975] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:24 2022] GET /identity/v3/role_assignments?user.id=687bbc1e0d174b78a51d01542d1fd03a&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=02a72f7b4ffa4327bc7c7db26907444d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:24.863450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 488/976] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:24.885743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:24.994865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1dd40-d7a3-4572-9579-81181f573aa5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:24.997691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:25.037022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:25.038306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.038594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.038873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.039694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:25.041084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:25.056196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.sql.core [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:28:25.056196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 16:28:25.056196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [parameters: {'user_id': '6aa4781be77e4b9d947ddb748f97d15f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 16:28:25.056196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80340) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:28:25.064743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:28:25.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application yield resource Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise self.value Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application session.commit() Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 16:28:25.066811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.session.flush() Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application compat.raise_( Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 16:28:25.068164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:28:25.069758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application result.read() Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:28:25.070743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [parameters: {'user_id': '6aa4781be77e4b9d947ddb748f97d15f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:25.071677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 16:28:25.072571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:25.073547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 489/977] 10.222.0.74 () {60 vars in 1131 bytes} [Thu Jun 16 16:28:24 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:25.077917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:25.190092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f193-020f-4fe9-abed-3debfeafae1a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:25.227418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:25.228542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.229042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.229494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.237451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:25.239178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:25.241187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-38c8fb4b-959d-49c3-8029-9f477a1ff6ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:25.257291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:25.258244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:28:25.259205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:25.262555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 489/978] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:25.268231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:25.376713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd0a4-9c10-4d27-9c9e-3cf24468f729 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:25.410997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:25.411843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.412282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.412658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.413621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:25.414025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:25.415791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620438ce-d1bf-4ad6-9018-01335dff1218 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:25.455582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 490/979] 10.222.0.74 () {56 vars in 1101 bytes} [Thu Jun 16 16:28:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:25.653138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410b040-7cce-4d97-ac51-8d850caab4a7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.653701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410b040-7cce-4d97-ac51-8d850caab4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.653977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410b040-7cce-4d97-ac51-8d850caab4a7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.655550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 490/980] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:25.661138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b0502-e17e-4aeb-b18d-8fbbf92c2fec None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.661624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b0502-e17e-4aeb-b18d-8fbbf92c2fec None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.662201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b0502-e17e-4aeb-b18d-8fbbf92c2fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.680087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d29b0502-e17e-4aeb-b18d-8fbbf92c2fec None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:25.681882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d29b0502-e17e-4aeb-b18d-8fbbf92c2fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:25.701109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:25.753864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 491/981] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:25.776534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410b040-7cce-4d97-ac51-8d850caab4a7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:25.887738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410b040-7cce-4d97-ac51-8d850caab4a7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:25.923124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:25.923872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:25.924224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:25.924570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:25.925367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:25.926864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd1b3a-574c-457f-aeb5-8f072d39664f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:25.942231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 491/982] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:25 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:26.659353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf9bd2-2924-4f8c-ad04-aef521e50fca None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:26.659906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf9bd2-2924-4f8c-ad04-aef521e50fca None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:26.659906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf9bd2-2924-4f8c-ad04-aef521e50fca None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:26.661320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 492/983] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:26.666898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:26.667228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:26.667510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:26.685083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:26.686300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:26.745055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:26.771098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 492/984] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:26.791024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf9bd2-2924-4f8c-ad04-aef521e50fca None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:26.898528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf9bd2-2924-4f8c-ad04-aef521e50fca None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:26.933056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:26.933965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:26.934258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:26.934531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:26.935217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:26.936591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-74fed633-6f9c-40d8-b463-e43b0b36bfd9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:26.948980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:26.950046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:26.950900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:26.955137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 493/985] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:26.956365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.068218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ee218-3474-491e-96ca-917efac6ba7a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.118488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.119192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.119475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.119748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.120469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:27.120784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.122486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.128969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 493/986] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:27.134171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.134438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0aae3-bec1-4279-834b-368cf8d5ffa0 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.134830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0aae3-bec1-4279-834b-368cf8d5ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.135259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0aae3-bec1-4279-834b-368cf8d5ffa0 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.136879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 494/987] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:27.142979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.143306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.143587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.163455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:27.164642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:27.179837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:27.190426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:27.240426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 495/988] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:27.250134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ab4d-e2c2-4900-ba3c-7ca5df2b328c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.263716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.290510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.291522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.291944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.292339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.293185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:27.293502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.295516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.306611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 494/989] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:27.313604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.374285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6beef3-f9fa-4d25-9aba-ea01e4a8f3a3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.411737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.412662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.413023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.413306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] PATH_INFO: `/v3/services/image` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.414022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.415395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.423100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f4fc3-417c-4f15-8c83-2204ac9479f0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:27.427029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:27.428250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:27.429133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 496/990] 10.222.0.74 () {56 vars in 1105 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:27.434226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.445120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:27.450668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:27.470208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.470978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.471202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.471466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.472266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] RBAC: Adding query filter params (interface=admin, service_id=cec58384051446688c1433cf86fce1d4, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:27.472578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.473954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.488311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 495/991] 10.222.0.74 () {56 vars in 1249 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/endpoints?service_id=cec58384051446688c1433cf86fce1d4&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:27.489462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.560206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee03d-c4f9-42ce-9630-23f76a87bd2e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.598624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ff8-19d0-4175-b065-f4b0de9ea1ae None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.599827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.600945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.601307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.601650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.602439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] RBAC: Adding query filter params (name=image) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:27.602818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.604250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.609593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 497/992] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:27.615037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:27.642929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.643625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.643903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.644176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.644858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.646228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679356d-4b79-4000-bb75-ef46a018c48b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.663092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 496/993] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:28:27.711496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:27.737691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc136-8289-4d49-8fa2-c5313c24299b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:27.778313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:27.779011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:27.779327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:27.779575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:27.780304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] RBAC: Adding query filter params (type=image) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:27.780613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:27.782010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178552df-2baa-4cd3-b2aa-ba42ba2dda9c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:27.791543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 498/994] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:28.103937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7005-18e1-4c13-bb8f-c8197e9f404b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.104447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7005-18e1-4c13-bb8f-c8197e9f404b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.104447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7005-18e1-4c13-bb8f-c8197e9f404b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.105954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 497/995] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:28.111237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.111499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.111807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.129359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:28.130558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:28.203166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 499/996] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:28.224081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f7005-18e1-4c13-bb8f-c8197e9f404b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:28.269469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:28.331995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f7005-18e1-4c13-bb8f-c8197e9f404b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:28.367639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:28.368363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.368625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.368914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.369566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:28.370978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:28.383135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:28.384146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:28.385000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:28.387390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 498/997] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:28.390522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:28.463578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:28.498823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46abce-ae0f-447b-8377-0e6044f9de25 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:28.538226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:28.538907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.539228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.539508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.540224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:28.540531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:28.542190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:28.557143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 500/998] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:28.558934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:28.669719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3de4b-52ef-4f63-bf38-65bab68f6b0b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:28.671976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:28.706525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:28.707216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.707505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.707788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.708496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:28.708810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:28.710949 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:28.716417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 499/999] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:28.722282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:28.830888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167b41e-5d97-42d0-a5fc-ddadff09ef25 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:28.871401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:28.872103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:28.872391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:28.872671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:28.873593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] RBAC: Adding query filter params (interface=public, service_id=ccfcc805c96a45a1a11677565e0bf9ae, region_id=RegionOne) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:28.873777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:28.875192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ebb6-1dde-4278-8cb7-86345b042552 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:28.876961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:28.887527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 501/1000] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:28 2022] GET /identity/v3/endpoints?service_id=ccfcc805c96a45a1a11677565e0bf9ae&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:28.890722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:28.994988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874df6b-1637-4cc5-be2c-4fd4b889c68b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:29.030104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:29.030765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:29.031074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:29.031431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:29.032128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:29.033493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27bf6e-b949-4f2e-a21c-fc8b8b451f6f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:29.047101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 500/1001] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:28 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:28:29.829451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4a09-557c-4f7c-9109-2ca266428413 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:29.829962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4a09-557c-4f7c-9109-2ca266428413 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:29.829962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4a09-557c-4f7c-9109-2ca266428413 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:29.834059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 502/1002] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:29.836798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:29.837210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:29.837377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:29.856811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:29.858053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:29.890504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:29.923296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c64d-d470-4ce8-995d-7954839db080 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:29.923648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c64d-d470-4ce8-995d-7954839db080 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:29.923912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1c64d-d470-4ce8-995d-7954839db080 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:29.925454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 503/1003] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:29.930708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:29.930708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:29.931581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:29.938135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 501/1004] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:29.948659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:29.949858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:29.961533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:30.020622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 504/1005] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:30.042652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:30.070309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aba617-3bf3-4c9e-92c2-edeece464997 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:30.103989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:30.104727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:30.105030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:30.105341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:30.108138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:30.108138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:30.120896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:30.122282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:30.123127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 502/1006] 10.222.0.74 () {56 vars in 1126 bytes} [Thu Jun 16 16:28:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:30.128688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:30.160875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc94dc-2d66-4a03-a829-799694db575e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:30.161260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:30.191477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:30.202153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:30.202827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:30.203879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:30.204149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:30.204920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:30.206247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:30.227187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 505/1007] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:30 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:30.256223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7049ac-94e0-4dc3-be14-94c122c29ea8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:30.305605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:30.306352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:30.306630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:30.306901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:30.307647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:30.307952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:30.309401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:30.320230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 503/1008] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:30.321549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:30.428834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c8e8-a3cd-4502-a29d-cf8c878d42ce None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:30.461708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:30.462384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:30.462669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:30.462947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:30.463687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:30.463997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:30.465367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a9a13-e0e3-4bf4-a930-561468f2272a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:30.470532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 506/1009] 10.222.0.74 () {56 vars in 1122 bytes} [Thu Jun 16 16:28:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:30.476666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:30.575337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:30.591132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d3c8-5fac-4a25-8257-6a62daa045b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:30.626242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:30.626906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:30.627217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:30.627490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:30.628223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:30.629597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17731c-0ceb-4b99-ae4c-7f13a2abadb9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:30.647529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 504/1010] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:31.162345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fac09d-6893-43b8-83db-1523b8e66a21 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.164030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fac09d-6893-43b8-83db-1523b8e66a21 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.164030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fac09d-6893-43b8-83db-1523b8e66a21 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.164574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 507/1011] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:31.169979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.170420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.170623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.188480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:31.189729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:31.271116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 505/1012] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:31.294659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fac09d-6893-43b8-83db-1523b8e66a21 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:31.357561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:31.410209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fac09d-6893-43b8-83db-1523b8e66a21 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:31.446941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:31.447841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.448122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.448397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.449105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:31.450490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:31.462995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:31.464028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:31.469930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 508/1013] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:31.472630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:31.557690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:31.591236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76d437-7b31-4a47-bd60-d1b1d4d08dcc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:31.628473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:31.629141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.629426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.629701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.630408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:31.630715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:31.632315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:31.642188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 506/1014] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:31.643828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:31.761290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7061241-3ed7-4b68-af80-e4150aedff3e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:31.779685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:31.799362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:31.800040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.800323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.800596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.801309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:31.801618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:31.803210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8590f90-aa15-42fb-aebf-ac9d5fa07430 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:31.808572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 509/1015] 10.222.0.74 () {56 vars in 1112 bytes} [Thu Jun 16 16:28:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:31.814801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:31.921608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd846e8-e391-4a6b-88b3-ae0be51d900c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:31.958824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:31.959567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:31.959852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:31.960131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:31.960873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:31.962258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b1f10-b2c3-47ef-9959-3f959c36a7c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:31.966533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:31.983088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 507/1016] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:32.374940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb8cac-4813-446e-89a2-138bdffb2f4a None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.375546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb8cac-4813-446e-89a2-138bdffb2f4a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.375546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb8cac-4813-446e-89a2-138bdffb2f4a None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.377083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 510/1017] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:32.382720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.383223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.383314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.400627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:32.401818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:32.479623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 508/1018] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:32.502369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb8cac-4813-446e-89a2-138bdffb2f4a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:32.626001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb8cac-4813-446e-89a2-138bdffb2f4a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:32.660372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:32.661167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.661445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.661720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.662563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:32.663958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-333c8066-05c0-4c36-901b-3872abc4a0b1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:32.675930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:32.676991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:32.678438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 511/1019] 10.222.0.74 () {56 vars in 1105 bytes} [Thu Jun 16 16:28:32 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:32.683309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:32.781635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afee45-e435-4aa2-9fe2-ce3916628492 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.781952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afee45-e435-4aa2-9fe2-ce3916628492 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.782204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afee45-e435-4aa2-9fe2-ce3916628492 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.783789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 512/1020] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:32.788892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.789167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.789423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.800570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1651-f6f8-4df4-920e-b87b461dfb4f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:32.806979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:32.808195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:32.821523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:32.841650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:32.842329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:32.842616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:32.842901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:32.843644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:32.843955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:32.845346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:32.850928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 509/1021] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:32.853951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:32.857264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:32.895620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 513/1022] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:32.913914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:32.968525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9299f-dddd-4d3f-b08e-70ac0ae545f4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:33.008279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:33.008969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:33.009245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:33.009661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:33.010219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:33.010529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:33.011924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:33.021708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 510/1023] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:33.022801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:33.031989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb66854-17ff-437d-a3e6-af055b2798e7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:33.074644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:33.075488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:33.075774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:33.076076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:33.076745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:33.077264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:33.078752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:33.080430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 514/1024] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:33.088606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:33.134629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:33.138992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a96e4-f5bf-44cc-bc70-89bbab7b8fa4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:33.177295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:33.178070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:33.178358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:33.178635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:33.179463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] RBAC: Adding query filter params (interface=public, service_id=d37eac1a5c464c5981164cee48923707, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:33.179778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:33.181254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:33.187201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 511/1025] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:28:33 2022] GET /identity/v3/endpoints?service_id=d37eac1a5c464c5981164cee48923707&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:33.193223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:33.204267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119215af-05c6-4d8c-9a02-4e40dfb6bd03 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:33.244237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:33.244925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:33.245209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:33.245486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:33.246489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:33.247962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea20143-8680-45c5-9c63-da981c62daf4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:33.286108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 515/1026] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:28:33 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:28:33.303365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dc28f-92b8-4edb-9a82-3aa3d4eeefd4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:33.316795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:33.345098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:33.345840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:33.346128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:33.346405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:33.347124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:33.348564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028cdd2-be58-4159-b29f-1028e72cd623 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:33.363386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 512/1027] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:28:33 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:28:34.289206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9900d0d-a328-4755-8ee3-7fa986b8434d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:34.289758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9900d0d-a328-4755-8ee3-7fa986b8434d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:34.289758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9900d0d-a328-4755-8ee3-7fa986b8434d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:34.291332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 516/1028] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:34.296465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:34.296892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:34.297034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:34.314665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:34.315900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:34.390528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 513/1029] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:34.414415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9900d0d-a328-4755-8ee3-7fa986b8434d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:34.520148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:34.529601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9900d0d-a328-4755-8ee3-7fa986b8434d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:34.570324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:34.571170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:34.571456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:34.571729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:34.582604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:34.591040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:34.591979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:34.598098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 517/1030] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:34.600455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:34.704786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a538ad3-f457-499a-b5c8-f6f3fdfe6038 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:34.707156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:34.740105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:34.740839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:34.741427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:34.741427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:34.742114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:34.742413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:34.743814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:34.752084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 514/1031] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:34.757791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:34.860739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831289af-89b5-4166-bebe-ca6613edfa99 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:34.898363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:34.899190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:34.899478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:34.899716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:34.910298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:34.911773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:34.930005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:34.930934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:34.931843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:34.931843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:34.931843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:34.931843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:34.935417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 518/1032] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:34.936895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.041013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb078-8858-4b6e-ad75-81e274ca2b1b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.074983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.075602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.075873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.076146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.076859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:35.077163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.078669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.091240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 515/1033] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:35.097010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.106169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.207657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b991576-cbe0-4d9e-90e0-098917c31911 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.244157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.244954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.245232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.245501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.253530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.255004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:35.271988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:35.273210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:35.274136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:35.278230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 519/1034] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:35.279087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.307711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.384188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9dd2-b98c-4299-96ca-ae19c7b6f158 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.424253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.424928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.425233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.425469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.426317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:35.426626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.428207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7e7e9-78e5-4d39-9ffb-7507bf2f7df6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.446293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 516/1035] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:35.451957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.456440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b84c-5d55-407a-9576-80e3b1abe7c5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.456767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b84c-5d55-407a-9576-80e3b1abe7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.457010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b84c-5d55-407a-9576-80e3b1abe7c5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.458494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 517/1036] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:35.464581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ee1f-1566-44b7-be08-e62603cce50a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.464931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ee1f-1566-44b7-be08-e62603cce50a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.465191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14ee1f-1566-44b7-be08-e62603cce50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.483419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-be14ee1f-1566-44b7-be08-e62603cce50a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:35.484667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-be14ee1f-1566-44b7-be08-e62603cce50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:35.528301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.563038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 518/1037] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:35.564630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f1f21-6c3d-404e-80f1-61b94306c404 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.564918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f1f21-6c3d-404e-80f1-61b94306c404 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.565191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f1f21-6c3d-404e-80f1-61b94306c404 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.565711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394ff06-d819-4e17-973f-c2c2b5d3fb00 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.567089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 519/1038] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:35.572727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.573015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.573292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.590227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:35.591443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:35.601175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.604821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.605491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.605776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.606051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.661499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 520/1039] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:35.662547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.685606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.687044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.752983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.768351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd04f4d-0273-4f41-9307-8be6bea703cd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.789819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 520/1040] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:35.790659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.804207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.805042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.805328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.805604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.806782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.808260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.809856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 521/1041] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:35.810614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.825005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.899935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b6fa8-0f8a-4b06-89e4-22981a03ba51 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.916205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389cc4-afc8-44f7-a264-16dfded0f906 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:35.935086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.935900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.936183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.936476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] PATH_INFO: `/v3/services/image` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.937121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.938494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:35.949879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:35.951093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:28:35.952095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:35.952980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:35.952980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:35.953155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 521/1042] 10.222.0.74 () {56 vars in 1105 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:35.953236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:35.953236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:35.953379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:35.964485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:35.966327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:35.984296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:35.985372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:35.986346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:35.986346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:35.986346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:35.986346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:35.986561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:35.990641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 522/1043] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:35.991396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.056028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:36.062204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a029b7-594f-4e96-92d5-9eeaeb350586 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.096844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8edf8-87cb-4b57-b419-c1b5e5678269 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.097455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.098126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.098409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.098680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.099418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.101165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.123486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 522/1044] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:28:35 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:36.124308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.135843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.137027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.137318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.138060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.138816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:36.139563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.141029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.153549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 523/1045] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:36.154860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.233290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:36.252689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ddb-6396-4a7a-aed6-5ef25bc930d2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.270985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e3fbf-8c08-4b47-8179-9dc28703cb8c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.292160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.293458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.293458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.293458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.294106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:36.294410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.295825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.304401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:36.310483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 523/1046] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:36.316774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.317256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.317506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.317795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.318067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.318887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] RBAC: Adding query filter params (type=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:36.319438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.320620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.326037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 524/1047] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:36.331985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.435145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fe8ca-19d4-4b05-9d28-a6caaa779a7d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.444746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f93fe1-d454-4062-a894-a31690ab5a93 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.475569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.476163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.476417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.476692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.483992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.484618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.485436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.485843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.486417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.486899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.488337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.489938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:36.499749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:36.500696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:36.501615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 524/1048] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:36.507293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.514885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 525/1049] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:28:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:36.517018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:36.627794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1c5cc-3eec-44e2-ba95-6cb6a4b3d988 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.671886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.672558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.672836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.673111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.673898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:36.674234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.675715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e42b53-35ae-477d-beb6-db0d34ba48e3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.689327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 525/1050] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:36.695742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:36.757847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:36.813420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812447b8-67c5-436f-acdc-fa506ef12f05 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:36.849262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:36.850058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:36.850346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:36.850616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:36.851491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] RBAC: Adding query filter params (role.id=02a72f7b4ffa4327bc7c7db26907444d, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:36.851806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:36.853247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a27ef-8a89-4ca6-8bb5-1c35c11674db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:36.859495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 526/1051] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:36 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=02a72f7b4ffa4327bc7c7db26907444d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:38.387658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090524-29f7-49c2-bc6f-898a82f299e3 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.387658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090524-29f7-49c2-bc6f-898a82f299e3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.387658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd090524-29f7-49c2-bc6f-898a82f299e3 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.388977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 526/1052] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:38.394953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.395453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.395529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.413923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:38.415171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:38.499292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 527/1053] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:38.529479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd090524-29f7-49c2-bc6f-898a82f299e3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:38.609347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:38.652083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd090524-29f7-49c2-bc6f-898a82f299e3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:38.687645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:38.688460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.688760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.689038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.700752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-08e2e17d-e059-4976-97fe-434c915866ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:38.708349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:38.709377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:38.716838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 527/1054] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:38.721802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:38.777831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:38.801295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d06d-5e21-4604-bb2d-53a790edfda3 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.801632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d06d-5e21-4604-bb2d-53a790edfda3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.801867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d06d-5e21-4604-bb2d-53a790edfda3 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.804057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 528/1055] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:38.808742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.809034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.809312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.827388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:38.828597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:38.845562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46b62-e65d-417f-bcd7-ab1789817a7c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:38.883375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:38.884052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:38.884331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:38.884599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:38.885336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:38.885635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:38.887013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:38.896935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 528/1056] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:38.903908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:38.911257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 529/1057] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:38.928336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:38.983197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.018407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499d85d-6652-4505-849c-923bf61bd03e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.022032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.038386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e423-1046-4630-b1c2-b1d2f58b7ca9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.070969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.071909 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.072186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.072456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] PATH_INFO: `/v3/users/c04923c779244ad494f7cbf27c226dac` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.087273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.087938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.088220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.088344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] RBAC: Authorizing `identity:get_user(user_id=c04923c779244ad494f7cbf27c226dac)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.088498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.089805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48938010-0cb7-4111-bf59-95fc6075bb24 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.096256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 529/1058] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:38 2022] GET /identity/v3/users/c04923c779244ad494f7cbf27c226dac => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:39.096256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf83ac-5e9f-4a32-95f8-4de6d870a2f9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.096256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf83ac-5e9f-4a32-95f8-4de6d870a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.096256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf83ac-5e9f-4a32-95f8-4de6d870a2f9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.096256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 530/1059] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:28:39.096975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf83ac-5e9f-4a32-95f8-4de6d870a2f9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.169448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.170843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5bfc2-0f9f-477a-8656-41db0c240e76 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.203497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf83ac-5e9f-4a32-95f8-4de6d870a2f9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.224066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.239201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.239855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.240140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.240583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.276468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.281360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 530/1060] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:28:39.282664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.282948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.283241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.298829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:39.300047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:39.322805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.324325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.368786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 531/1061] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:39.369627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.436898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 531/1062] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:28:39.437753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.479626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.498795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a7aee-d430-4a5f-be5b-a6e91f8de11c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.499607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.538187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce02ac-58be-4f89-b39e-13243350e53d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.540049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.540891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.541174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.541448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.556184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.557647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.572989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.573803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.574081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.574352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:39.574510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:39.575377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 532/1063] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:39.576973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.586586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:39.594700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:39.595696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:39.596589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 532/1064] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:39.597634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.684552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c58aa-0dc3-49de-98e2-7654535871cd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.708280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2062-dd6b-40ae-8c90-91fb1ee9ef80 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.729777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.730645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.730932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.731241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.732433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.733844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.733982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.735597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 533/1065] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:39.736357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.738993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:39.748399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.749069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.749348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.749624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.750354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:39.750628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.752044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.764404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 533/1066] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:39.765204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.859370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180dd30-90a2-4822-a863-583096e43fe5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.877439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6a20-ce58-4068-9927-37660f86f6f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:39.901529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.902218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.902513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.902769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.903572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:39.903903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.905275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.919519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 534/1067] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:39.920929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:39.922456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:39.923171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:39.923457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:39.923728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:39.924572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=ecc36e6d998c4bf885d66239bd779559, user.id=c04923c779244ad494f7cbf27c226dac) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:39.924886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:39.926338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:39.939079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 534/1068] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/role_assignments?user.id=c04923c779244ad494f7cbf27c226dac&scope.project.id=ecc36e6d998c4bf885d66239bd779559&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:40.014176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:40.060444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c6eb7-000a-43b5-8511-ccd7d7987acb None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.098048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.098852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.099159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.099439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.112033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.113487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:40.131490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:40.132423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:40.133292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:40.133292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:40.133292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:40.133292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:40.133292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 535/1069] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:28:40.138559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:40.186739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:40.241923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1de3f6-4bf8-4455-9d11-aa620018a14b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.277125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.277701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.277979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.278246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.278964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:40.279295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.280657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.292910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 535/1070] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:40.298613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:40.403571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7143-ee72-4b04-8868-d84696bd6d71 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.420577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:40.440766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.441564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.441846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.442115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.450189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.451661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:40.465612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:40.466553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:40.467501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 536/1071] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:28:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:40.472794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:40.586826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970c8b1-9750-4662-96de-1cdfb123effd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.594748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:40.623458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.624141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.624428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.624702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.625547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:40.625858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.627325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.645047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 536/1072] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:28:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:40.650855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:40.758298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8579cfd-fe0d-4e7a-b161-afe0e59dc650 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.794484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.795321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.795604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.795895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.807148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.808566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:40.826163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:40.827227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:40.828116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:40.828116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:40.828116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:40.828116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:40.828116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 537/1073] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:40.833341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:40.944289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286cfcc-d90d-42a8-a9ea-fad8412381d7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:40.979311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:40.979965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:40.980246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:40.980710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:40.981422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:40.981723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:40.983084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:40.995001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 537/1074] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:41.001211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:41.032400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:41.108337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f66cd-9298-47c4-a58b-e1791ab4f18a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:41.143069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:41.144058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:41.144336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:41.144592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:41.151706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:41.153145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:41.168167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:41.169200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:41.170084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 538/1075] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:41.175397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:41.204603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:41.283356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbd836-218a-4382-8db7-c309880eae6d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:41.317742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:41.318423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:41.318715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:41.318989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:41.319804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:41.320117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:41.321571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e766c3-6d71-4e1e-88e5-3dcdc1dd6213 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:41.333772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 538/1076] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:41.340176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:41.431535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:41.444115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e55621-cbcc-4625-90ee-0273a792b0d0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:41.479243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:41.480124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:41.480401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:41.480668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/02a72f7b4ffa4327bc7c7db26907444d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:41.482967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=b6297dc028a44750a69132257e0aede1, role_id=02a72f7b4ffa4327bc7c7db26907444d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:41.484435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3973d45-0c5b-45f2-82ec-bd5026c43a5b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:41.499122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 539/1077] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:41 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/b6297dc028a44750a69132257e0aede1/roles/02a72f7b4ffa4327bc7c7db26907444d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:42.055019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd78b-f61b-443b-ab1e-8d5c5e285774 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.055594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd78b-f61b-443b-ab1e-8d5c5e285774 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.055758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccd78b-f61b-443b-ab1e-8d5c5e285774 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.057151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 539/1078] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:42.062844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.063325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.063418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.080579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:42.081768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:42.151867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 540/1079] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:28:42.174112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd78b-f61b-443b-ab1e-8d5c5e285774 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:42.264878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:42.282995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccd78b-f61b-443b-ab1e-8d5c5e285774 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:42.318768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:42.319883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.320320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.320753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.332070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:42.339954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:42.340864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:42.341678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 540/1080] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:42.347724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:42.428777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:42.452377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a88b0d-8014-4023-b415-105c938a7810 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:42.488017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:42.488701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.488986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.489254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.490126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:42.490433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:42.492035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:42.501392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 541/1081] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:42.507099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:42.611307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d069b70-1b17-4c00-bc77-1da6e742f1fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:42.649426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:42.650314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.650597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.650876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] PATH_INFO: `/v3/users/c04923c779244ad494f7cbf27c226dac` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.652157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=c04923c779244ad494f7cbf27c226dac)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:42.653599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:42.655263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 541/1082] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:42 2022] GET /identity/v3/users/c04923c779244ad494f7cbf27c226dac => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:42.661171 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:42.760804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53b206-41c5-4663-b850-51601c44ac58 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:42.795460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:42.796324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.796605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.796884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.798106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:42.799588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90a94-acad-43c1-92e8-0570951448af None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:42.801494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 542/1083] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:42 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:42.807872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:42.824860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:42.911212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfb98e-53f9-4d8c-8797-6b01beb7a1e9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:42.944050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:42.944977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:42.945260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:42.945536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559/users/c04923c779244ad494f7cbf27c226dac/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:42.947556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] RBAC: Authorizing `identity:create_grant(project_id=ecc36e6d998c4bf885d66239bd779559, user_id=c04923c779244ad494f7cbf27c226dac, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:42.949005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9049c-86ef-4e5b-9745-1babac2930dd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:42.963147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 542/1084] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:42 2022] PUT /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559/users/c04923c779244ad494f7cbf27c226dac/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:28:43.606191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3c14-c8ab-4116-a77f-d8546f8c3a0a None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:43.606736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3c14-c8ab-4116-a77f-d8546f8c3a0a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:43.606736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea3c14-c8ab-4116-a77f-d8546f8c3a0a None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:43.608311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 543/1085] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:43.613656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:43.614088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:43.614280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:43.632294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:43.633479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:43.705069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 543/1086] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:43.728704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea3c14-c8ab-4116-a77f-d8546f8c3a0a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:43.741134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:43.838518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea3c14-c8ab-4116-a77f-d8546f8c3a0a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:43.873182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:43.873994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:43.874281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:43.874550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:43.885474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:43.893622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:43.894771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:28:43.895851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:43.899414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 544/1087] 10.222.0.74 () {56 vars in 1120 bytes} [Thu Jun 16 16:28:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:43.901035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:43.903278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.003618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b8ca3-b3a7-4d3d-956f-33f9a255c0b2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.040609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.041282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.041563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.041843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.042622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:44.042925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.044325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:44.052854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 544/1088] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:28:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:44.058996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:44.129619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.166205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e856f-104a-4fd7-90ad-8122e00aed95 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.202578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.203418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.203706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.203984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.214795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.216318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:44.235035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:44.236079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:44.237069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:44.237069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:44.237069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:44.237069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:44.239196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 545/1089] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:44.242400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:44.306429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.344428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe2fd-a146-41e0-8ad4-8a72c299504c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.378030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.378693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.378979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.379289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.380065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:44.380326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.381756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:44.393341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 545/1090] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:44.399289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:44.498191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230754b-c131-4a27-a35f-588ba4208663 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.517887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.533237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.534034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.534279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.534586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.542487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.543956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:28:44.559972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:44.560952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:44.562085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 546/1091] 10.222.0.74 () {56 vars in 1093 bytes} [Thu Jun 16 16:28:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:44.566946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:44.672247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f3e2e-4a75-489a-9b45-6e7669f62105 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.696732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.707913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.708597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.708884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.709193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.710051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:44.710358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.711834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f667b3-0e80-47de-8c0e-347a2a158167 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:44.729373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 546/1092] 10.222.0.74 () {56 vars in 1133 bytes} [Thu Jun 16 16:28:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:44.735539 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:44.842786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5bd67-4975-4528-a0da-b1c418b9e241 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:44.879504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:44.880160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:44.880443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:44.880724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:44.933339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:44.954497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:44.956002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.060416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e3df-8bd9-48cc-8900-70db46b8866e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.060748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e3df-8bd9-48cc-8900-70db46b8866e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.060975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77e3df-8bd9-48cc-8900-70db46b8866e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.062338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 547/1093] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:45.067963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 547/1094] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:45.068091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.068091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.068272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.073282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.086503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:45.087736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:45.155733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.157030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 548/1095] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:45.175319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86decf6-7e64-4f5a-b6d3-ef5bb91a1d75 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.180492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.211891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.212610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.212889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.213165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.223821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.225510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:28:45.243317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:45.244324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:45.245196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:28:45.245196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:28:45.245196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:28:45.245196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:45.245196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 548/1096] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:45.250500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.282485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599bc9a-1725-4026-a23c-de70873327fe None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.295657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.320152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.320962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.321577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.332071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:28:45.339699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:45.340838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:28:45.341679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 549/1097] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:45.346779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.357477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd08dc-2e12-49fe-a9e7-23cc2f33692e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.386940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.392682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.393349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.393627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.393893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.394606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:45.394912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.396298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.407655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 549/1098] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:45.414014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.452793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff27f-063e-43e9-825f-f9903ab84208 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.488426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.489025 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.489311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.489587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.490388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:45.490908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.492339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.501351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 550/1099] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:45.507348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.517192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5f5a-848b-4ce4-b677-1c5495f7af46 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.542093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.554776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.555630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.555909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.556181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.563281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.564729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:28:45.579017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:28:45.579999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:28:45.581430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 550/1100] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:28:45.586146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.613882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6ffc-a10a-4177-bb3e-1a32a7af207d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.639349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.648655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.649549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.649837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.650115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] PATH_INFO: `/v3/users/c04923c779244ad494f7cbf27c226dac` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.651360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] RBAC: Authorizing `identity:get_user(user_id=c04923c779244ad494f7cbf27c226dac)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.652841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.654475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 551/1101] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/users/c04923c779244ad494f7cbf27c226dac => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:45.660062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.687529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c000928-7f18-4c83-a4c7-d78ac5a8a6d6 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.721121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.721750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.722032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.722302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.723102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:45.723415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.724909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.736977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 551/1102] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:45.743340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:45.761526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017c089-b7b5-4d54-88ce-5b0ab4c6164f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.763242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.799388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.800045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.800332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.800607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.850316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c4a67-ffe5-4801-a49b-45116f26b474 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:45.863728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:45.871449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.873228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.889266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:45.889937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:45.890221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:45.890497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:45.891364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] RBAC: Adding query filter params (role.id=02a72f7b4ffa4327bc7c7db26907444d, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=b6297dc028a44750a69132257e0aede1) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:45.891680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:45.893106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:45.900194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 552/1103] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/role_assignments?user.id=b6297dc028a44750a69132257e0aede1&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=02a72f7b4ffa4327bc7c7db26907444d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:28:45.982072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 552/1104] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:45.988211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:46.098760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75cd71-cd65-4ef4-9560-8e079d6493cf None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:46.133944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:46.134831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:46.135135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:46.135403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:46.136603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:46.138031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eb96e-f17e-431a-b906-768175c67b6b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:46.139817 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 553/1105] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:45 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:46.146168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:46.150001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:46.262163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4ac9c-d743-431e-bf8a-94d06cfe513a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:46.298539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:46.299316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:46.299576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:46.299847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:46.300672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=ecc36e6d998c4bf885d66239bd779559, user.id=c04923c779244ad494f7cbf27c226dac) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:46.300984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:46.302410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231816a-6ec1-458a-b772-907f93a08399 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:46.308896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 553/1106] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:46 2022] GET /identity/v3/role_assignments?user.id=c04923c779244ad494f7cbf27c226dac&scope.project.id=ecc36e6d998c4bf885d66239bd779559&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:48.464850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396bad-a896-4876-839c-f130683179f3 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:48.465433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396bad-a896-4876-839c-f130683179f3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:48.465433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396bad-a896-4876-839c-f130683179f3 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:48.466782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 554/1107] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:48.473296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:48.474014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:48.474189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:48.492647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:48.493839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:48.556217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:48.564866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 554/1108] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:48.589623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396bad-a896-4876-839c-f130683179f3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:48.649735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:48.716409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396bad-a896-4876-839c-f130683179f3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:48.754841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:48.755665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:48.756030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:48.756359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:48.757619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:48.759079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ba587-f485-46d9-8ea4-89517df460bd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:48.760727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 555/1109] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:28:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:48.769780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:48.880321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a5a1c-b934-45e0-bd65-c20e1567b28e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:48.920441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:48.921503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:48.921962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:48.922392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:48.923638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:48.926182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8591b-b7e0-4ef5-b867-a0829c55b03c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:48.947734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 555/1110] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:28:48 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:28:51.107784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da840-1013-49ee-b7e1-8387b1855ce0 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:51.108387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da840-1013-49ee-b7e1-8387b1855ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:51.108387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da840-1013-49ee-b7e1-8387b1855ce0 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:51.109722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 556/1111] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:51.115232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:51.115687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:51.115830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:51.134789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:51.136024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:51.200646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:51.221986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 556/1112] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:51.246494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da840-1013-49ee-b7e1-8387b1855ce0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:51.356814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da840-1013-49ee-b7e1-8387b1855ce0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:51.370902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:51.401346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:51.402872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:51.403449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:51.403960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] PATH_INFO: `/v3/roles/4169efc51b3041918162437c5960c781` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:51.406576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] RBAC: Authorizing `identity:get_role(role_id=4169efc51b3041918162437c5960c781)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:51.408942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:51.411477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 557/1113] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:51 2022] GET /identity/v3/roles/4169efc51b3041918162437c5960c781 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:51.420699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:51.540532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc627e-7620-4f12-b557-67104a0715ae None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:51.577482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:51.578359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:51.578640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:51.578916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] PATH_INFO: `/v3/users/4f73e4795c8f4288bfe925403eac686e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:51.594833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] RBAC: Authorizing `identity:get_user(user_id=4f73e4795c8f4288bfe925403eac686e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:51.596334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:51.597966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 557/1114] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:51 2022] GET /identity/v3/users/4f73e4795c8f4288bfe925403eac686e => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:51.604686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:51.734652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526973c-2fb0-4041-bcd1-8956e368bfbe None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:51.773335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:51.773987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:51.774272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:51.774548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:51.830967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:51.855822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:51.857408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:51.964044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 558/1115] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:51.971105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:52.024813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:52.095677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfffa0-7f45-4c23-85c2-0a06630e7afe None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:52.133159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:52.134010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:52.134294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:52.134553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:52.135872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:52.137352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf65db-dfc8-4dc2-ab03-301637be173f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:52.139084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 558/1116] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:51 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:52.145801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:52.234311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:52.258851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fee3-f60d-4ace-be84-2f8778f1de40 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:52.297160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:52.297739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:52.298023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:52.298293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:52.299159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=ecc36e6d998c4bf885d66239bd779559, user.id=4f73e4795c8f4288bfe925403eac686e) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:52.299469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:52.300904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d36c-8571-401d-8f80-8b8b96c030fc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:52.306545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 559/1117] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:52 2022] GET /identity/v3/role_assignments?user.id=4f73e4795c8f4288bfe925403eac686e&scope.project.id=ecc36e6d998c4bf885d66239bd779559&role.id=4169efc51b3041918162437c5960c781 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:54.387229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3fd-a2e4-4698-be12-d2f76fc51f67 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:54.387791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3fd-a2e4-4698-be12-d2f76fc51f67 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:54.387791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3fd-a2e4-4698-be12-d2f76fc51f67 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:54.389151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 559/1118] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:54.395550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:54.396282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:54.396427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:54.423725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:54.425582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:54.504818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 560/1119] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:54.534088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd3fd-a2e4-4698-be12-d2f76fc51f67 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:54.645365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd3fd-a2e4-4698-be12-d2f76fc51f67 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:54.654855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:54.689869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:54.690731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:54.691026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:54.691335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] PATH_INFO: `/v3/roles/4169efc51b3041918162437c5960c781` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:54.692885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] RBAC: Authorizing `identity:get_role(role_id=4169efc51b3041918162437c5960c781)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:54.694793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:54.696210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 560/1120] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:54 2022] GET /identity/v3/roles/4169efc51b3041918162437c5960c781 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:54.704888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:54.781382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:54.809512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d68c-97a6-4164-800a-f5c4cc555c9a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:54.845465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:54.846329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:54.846616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:54.846892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] PATH_INFO: `/v3/users/4f73e4795c8f4288bfe925403eac686e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:54.848155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] RBAC: Authorizing `identity:get_user(user_id=4f73e4795c8f4288bfe925403eac686e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:54.849596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:54.851546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 561/1121] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:54 2022] GET /identity/v3/users/4f73e4795c8f4288bfe925403eac686e => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:54.857821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:54.988522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc325f-defc-4bb0-9e4f-b8b6798c11b2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:55.033525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:55.034816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:55.035221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:55.035632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:55.037552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:55.038997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f4584-a9fc-42ac-8bf0-a848688758fc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:55.040754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 561/1122] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:54 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:55.047027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:55.161703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cebc3-4507-4063-81a1-e82ef1d3f1cd None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:55.196635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:55.197639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:55.197909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:55.198165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559/users/4f73e4795c8f4288bfe925403eac686e/roles/4169efc51b3041918162437c5960c781` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:55.200570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecc36e6d998c4bf885d66239bd779559, user_id=4f73e4795c8f4288bfe925403eac686e, role_id=4169efc51b3041918162437c5960c781)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:55.202008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1438ac7-fe11-4393-ad9f-9d2b78e91318 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:55.217773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 562/1123] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:28:55 2022] PUT /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559/users/4f73e4795c8f4288bfe925403eac686e/roles/4169efc51b3041918162437c5960c781 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:28:57.302782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4d1-6e5f-47ef-ae27-2ec05b84a236 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:57.303376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4d1-6e5f-47ef-ae27-2ec05b84a236 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:57.303579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5e4d1-6e5f-47ef-ae27-2ec05b84a236 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:57.305095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 562/1124] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:28:57.311388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:57.312097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:57.312248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:57.320311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:57.334400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:28:57.335762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:57.405107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 563/1125] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:28:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:28:57.428583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e4d1-6e5f-47ef-ae27-2ec05b84a236 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:57.481717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:57.552132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5e4d1-6e5f-47ef-ae27-2ec05b84a236 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:57.589443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:57.590386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:57.590661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:57.590929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] PATH_INFO: `/v3/roles/4169efc51b3041918162437c5960c781` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:57.592545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] RBAC: Authorizing `identity:get_role(role_id=4169efc51b3041918162437c5960c781)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:57.593973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:57.595305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 563/1126] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:57 2022] GET /identity/v3/roles/4169efc51b3041918162437c5960c781 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:57.602485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:57.711568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e7c99-584a-4c1f-8fa2-4786ae9dc572 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:57.745000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:57.751277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:57.752157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:57.752435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:57.752708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] PATH_INFO: `/v3/users/4f73e4795c8f4288bfe925403eac686e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:57.753940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] RBAC: Authorizing `identity:get_user(user_id=4f73e4795c8f4288bfe925403eac686e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:57.755407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:57.757073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 564/1127] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:28:57 2022] GET /identity/v3/users/4f73e4795c8f4288bfe925403eac686e => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:57.763067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:57.866674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96b85-975e-4c61-a5f0-eb334ae84584 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:57.882615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:57.907184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:57.907838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:57.908108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:57.908383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:57.986931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:57.988427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:58.109765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 564/1128] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:28:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:28:58.119319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:58.231563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200b55d-839c-458c-83a3-a855729ba0bc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:58.273114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:58.273975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:58.274250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:58.274518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] PATH_INFO: `/v3/projects/ecc36e6d998c4bf885d66239bd779559` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:58.275808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ecc36e6d998c4bf885d66239bd779559)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:58.277242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523960c0-b74a-4738-8eed-1fa36f5b03a1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:58.279077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 565/1129] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:28:58 2022] GET /identity/v3/projects/ecc36e6d998c4bf885d66239bd779559 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:28:58.285490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:58.301396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:58.393270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61e69-f4bd-4b23-8d08-e29a49aeb5ef None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:58.431336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:58.431980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:58.432260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:58.432530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:58.433354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] RBAC: Adding query filter params (role.id=4169efc51b3041918162437c5960c781, scope.project.id=ecc36e6d998c4bf885d66239bd779559, user.id=4f73e4795c8f4288bfe925403eac686e) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:28:58.433660 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:58.435107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f11e-41bd-4862-8113-ce84f10b976b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:58.441213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 565/1130] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:28:58 2022] GET /identity/v3/role_assignments?user.id=4f73e4795c8f4288bfe925403eac686e&scope.project.id=ecc36e6d998c4bf885d66239bd779559&role.id=4169efc51b3041918162437c5960c781 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:00.559665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cecf9-a8c8-4a7b-a252-8bdae3c2e421 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:00.560168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cecf9-a8c8-4a7b-a252-8bdae3c2e421 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:00.560168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cecf9-a8c8-4a7b-a252-8bdae3c2e421 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:00.561604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 566/1131] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:00.567896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:00.568625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:00.568787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:00.591381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:00.592587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:00.666910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 566/1132] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:00.691189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cecf9-a8c8-4a7b-a252-8bdae3c2e421 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:00.769831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:00.806361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cecf9-a8c8-4a7b-a252-8bdae3c2e421 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:00.844066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:00.844857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:00.845143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:00.845419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:00.846602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:00.848072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb896ee-1ff9-4e0f-97f1-838316c9e0f2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:00.849695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 567/1133] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:29:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:00.856454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:00.864656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:00.965896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658476-86aa-4f5d-946a-7cc5e94df84a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:01.001895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:01.002545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:01.002828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:01.003113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:01.003825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:01.005264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba143fd-8666-458e-9199-992c37a8ac42 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:01.034796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 567/1134] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:29:00 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:03.220901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ac9e-f20f-4a3d-aaa3-0ca5af6dbea6 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:03.221506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ac9e-f20f-4a3d-aaa3-0ca5af6dbea6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:03.221506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4ac9e-f20f-4a3d-aaa3-0ca5af6dbea6 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:03.222836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 568/1135] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:03.229424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:03.229952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:03.229952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:03.248468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:03.249680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:03.312743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:03.329275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 568/1136] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:03.354937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4ac9e-f20f-4a3d-aaa3-0ca5af6dbea6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:03.471241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4ac9e-f20f-4a3d-aaa3-0ca5af6dbea6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:03.511341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:03.512127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:03.512430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:03.512699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:03.513963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:03.515433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36912c94-7ff9-4f08-b6f6-fed82c108c28 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:03.517075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 569/1137] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:03.523299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:03.630577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa267-02ce-4c02-8cdc-af1687f47a69 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:03.676568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:03.677439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:03.677806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:03.678231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:03.679097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:03.680536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261d499-bb12-424c-aaf4-513f29a4b542 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:03.702936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 569/1138] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:29:03 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:05.864557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b477d4-a982-439b-93e3-04208f58e772 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:05.865104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b477d4-a982-439b-93e3-04208f58e772 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:05.865104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b477d4-a982-439b-93e3-04208f58e772 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:05.866408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 570/1139] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:05.873285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:05.874005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:05.874149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:05.897963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:05.899217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:05.945603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:05.986992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 570/1140] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:06.011888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b477d4-a982-439b-93e3-04208f58e772 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:06.016296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:06.142040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b477d4-a982-439b-93e3-04208f58e772 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:06.179364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:06.180160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:06.180439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:06.180713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:06.192045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:06.199669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:06.200802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:06.202010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 571/1141] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:06.209596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:06.316264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1b242-580e-4fcc-abcf-ee4f7842d879 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:06.354426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:06.355429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:06.355803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:06.356181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:06.357314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:06.357750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:06.360310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:06.371030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 571/1142] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:06.377533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:06.446639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:06.490389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1572a-0c19-4203-a885-6c7c7fb93bc1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:06.534404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:06.535324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:06.535614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:06.535902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] PATH_INFO: `/v3/users/85c4ce6a25394a8fbdf361a474f40e6b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:06.550224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] RBAC: Authorizing `identity:get_user(user_id=85c4ce6a25394a8fbdf361a474f40e6b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:06.551724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:06.553436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 572/1143] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:06 2022] GET /identity/v3/users/85c4ce6a25394a8fbdf361a474f40e6b => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:06.559625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:06.596166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:06.665616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfff4-42e2-449d-bfb1-bf7e4c67122a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:06.702996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:06.703596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:06.703876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:06.704152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:06.778788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:06.780268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:06.814521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:06.891108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 572/1144] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:29:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:29:06.900190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:07.007853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c5e0d-dd10-480d-a0cc-5e6ea060b3a1 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:07.030957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:07.054687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:07.055992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:07.056360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:07.056712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] PATH_INFO: `/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:07.058696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] RBAC: Authorizing `identity:get_project(project_id=5dcef904050c4d9d99c8d9c570e74d7f)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:07.061045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ab9db-e7f1-4d7e-b57f-073c808f41e3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:07.063741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 573/1145] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:06 2022] GET /identity/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:07.070911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:07.190744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89e790-c9a1-4cf4-bccc-d20ec2a79f7e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:07.226857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:07.227544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:07.227832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:07.228137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:07.228965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=5dcef904050c4d9d99c8d9c570e74d7f, user.id=85c4ce6a25394a8fbdf361a474f40e6b) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:07.229281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:07.230729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51598a1-ce2b-4cf2-b8c0-8b65d9a66c83 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:07.233724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:07.236673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 573/1146] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:29:07 2022] GET /identity/v3/role_assignments?user.id=85c4ce6a25394a8fbdf361a474f40e6b&scope.project.id=5dcef904050c4d9d99c8d9c570e74d7f&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:29:09.318587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c3d9-4961-4637-a941-f20b2cabc9ac None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:09.319502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c3d9-4961-4637-a941-f20b2cabc9ac None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:09.320421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33c3d9-4961-4637-a941-f20b2cabc9ac None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:09.322668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 574/1147] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:09.328931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:09.329654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:09.329806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:09.350089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:09.351407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:09.424244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 574/1148] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:09.447686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c3d9-4961-4637-a941-f20b2cabc9ac None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:09.563385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33c3d9-4961-4637-a941-f20b2cabc9ac None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:09.599803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:09.600560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:09.600843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:09.601120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:09.611669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:09.619567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:09.620624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:09.621497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 575/1149] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:09.629983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:09.740522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3b0f-e843-496c-bacd-0e05b39c8608 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:09.771256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:09.779094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:09.779786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:09.780073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:09.780341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:09.781103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:09.781411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:09.782807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:09.791656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 575/1150] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:09.799893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:09.854182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:09.908494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb0697-38c6-4660-b0c2-2db5e1104cc9 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:09.943950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:09.944999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:09.945281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:09.945554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] PATH_INFO: `/v3/users/85c4ce6a25394a8fbdf361a474f40e6b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:09.946870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] RBAC: Authorizing `identity:get_user(user_id=85c4ce6a25394a8fbdf361a474f40e6b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:09.948358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:09.950006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 576/1151] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:09 2022] GET /identity/v3/users/85c4ce6a25394a8fbdf361a474f40e6b => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:09.956162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:10.066015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ef904-04b0-4f91-a2ac-d32833bc0f02 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:10.102836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:10.103725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:10.104010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:10.104351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] PATH_INFO: `/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:10.105578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] RBAC: Authorizing `identity:get_project(project_id=5dcef904050c4d9d99c8d9c570e74d7f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:10.107020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b83b6a-2880-4d37-980e-1b7289e9c3fb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:10.109224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 576/1152] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:09 2022] GET /identity/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:10.118846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:10.229004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f1aeb-710d-483e-93fb-56e944cb31f3 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:10.248744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:10.269965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:10.270871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:10.271200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:10.271484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] PATH_INFO: `/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f/users/85c4ce6a25394a8fbdf361a474f40e6b/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:10.273982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] RBAC: Authorizing `identity:create_grant(project_id=5dcef904050c4d9d99c8d9c570e74d7f, user_id=85c4ce6a25394a8fbdf361a474f40e6b, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:10.275469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85985043-9477-4ca2-8ec5-11193b3c166c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:10.290197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 577/1153] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:29:10 2022] PUT /identity/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f/users/85c4ce6a25394a8fbdf361a474f40e6b/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:12.387310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f34b91-23ce-4994-a83c-4786b77b06cc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:12.387897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f34b91-23ce-4994-a83c-4786b77b06cc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:12.387897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f34b91-23ce-4994-a83c-4786b77b06cc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:12.389225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 577/1154] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:12.395675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:12.396115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:12.396251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:12.417213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:12.418409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:12.492192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 578/1155] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:12.521856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f34b91-23ce-4994-a83c-4786b77b06cc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:12.599278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:12.646911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f34b91-23ce-4994-a83c-4786b77b06cc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:12.684415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083289a-8982-4122-81e3-c190582a322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:12.685201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083289a-8982-4122-81e3-c190582a322e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:12.685474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083289a-8982-4122-81e3-c190582a322e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:12.685746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083289a-8982-4122-81e3-c190582a322e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:12.696661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083289a-8982-4122-81e3-c190582a322e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0083289a-8982-4122-81e3-c190582a322e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:12.704234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:12.705295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:29:12.706310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 578/1156] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:12.713101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:12.821948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485ce74-d807-4a4b-b8b1-a0e2646a505b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:12.831303 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:12.869394 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:12.870060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:12.870349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:12.870627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:12.871408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:12.871720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:12.873424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:12.881866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 579/1157] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:12.890947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083289a-8982-4122-81e3-c190582a322e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:13.017606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083289a-8982-4122-81e3-c190582a322e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:13.041221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:13.057819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:13.058703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:13.058985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:13.059291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] PATH_INFO: `/v3/users/85c4ce6a25394a8fbdf361a474f40e6b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:13.060540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] RBAC: Authorizing `identity:get_user(user_id=85c4ce6a25394a8fbdf361a474f40e6b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:13.061999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:13.063652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 579/1158] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:12 2022] GET /identity/v3/users/85c4ce6a25394a8fbdf361a474f40e6b => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:13.069974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:13.179383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17a95-604c-4996-9489-ff20e5e1ee13 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:13.215803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:13.216447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:13.216729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:13.217011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:13.268006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:13.297775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:13.299265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:13.400653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 580/1159] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:29:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:29:13.410130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:13.515884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6df6-d1c7-466b-b845-6315cdad42c6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:13.552941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:13.553826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:13.554134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:13.554400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] PATH_INFO: `/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:13.555626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] RBAC: Authorizing `identity:get_project(project_id=5dcef904050c4d9d99c8d9c570e74d7f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:13.557070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808029b6-18f6-477c-a60d-ec3c4856eeb9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:13.558767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 580/1160] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:13 2022] GET /identity/v3/projects/5dcef904050c4d9d99c8d9c570e74d7f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:13.565097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:13.635679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:13.667740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b85e-410a-4c67-85a1-49cde28019b2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:13.707146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:13.707804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:13.708082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:13.708354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:13.709174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=5dcef904050c4d9d99c8d9c570e74d7f, user.id=85c4ce6a25394a8fbdf361a474f40e6b) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:13.709529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:13.711078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d4b06-906b-4ab2-af1d-001d9d1994e4 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:13.717741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 581/1161] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:29:13 2022] GET /identity/v3/role_assignments?user.id=85c4ce6a25394a8fbdf361a474f40e6b&scope.project.id=5dcef904050c4d9d99c8d9c570e74d7f&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:15.840275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa590-43ee-4308-8543-a33a734372d9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:15.841080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa590-43ee-4308-8543-a33a734372d9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:15.841080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa590-43ee-4308-8543-a33a734372d9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:15.843310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 581/1162] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:15.848707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:15.849142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:15.849233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:15.868963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:15.870169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:15.949224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 582/1163] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:29:15.973184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fa590-43ee-4308-8543-a33a734372d9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:16.040361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:16.104272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fa590-43ee-4308-8543-a33a734372d9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:16.140837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:16.141510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:16.141790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:16.142068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:16.215677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:16.217134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:16.280077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:16.329287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 582/1164] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:29:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:29:16.339466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:16.416060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:16.455038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c441f3-eead-46af-b274-e51662eba6c5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:16.489826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:16.490616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:16.490978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:16.491285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:16.498355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:16.499908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-8062d9c9-4a70-4f61-bdf9-4721a2c7c4f6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:16.513911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:16.515103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:16.516041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 583/1165] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:29:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:16.522131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:16.633179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0f376-55fb-483d-9029-94aecfdd892a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:16.667861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:16.668446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:16.668738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:16.669003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:16.669720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:16.670031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:16.671425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967e5c-86f5-440f-9bb6-3b655122fc66 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:16.678768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 583/1166] 10.222.0.74 () {56 vars in 1115 bytes} [Thu Jun 16 16:29:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:18.801996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86de9-d919-45fd-9e36-0ac9913b3ad4 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:18.802833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86de9-d919-45fd-9e36-0ac9913b3ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:18.802833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc86de9-d919-45fd-9e36-0ac9913b3ad4 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:18.805030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 584/1167] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:18.813566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210e49-cb80-4aeb-af46-0e7a8ee6179c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:18.813985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210e49-cb80-4aeb-af46-0e7a8ee6179c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:18.814178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210e49-cb80-4aeb-af46-0e7a8ee6179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:18.836032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6210e49-cb80-4aeb-af46-0e7a8ee6179c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:18.837256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6210e49-cb80-4aeb-af46-0e7a8ee6179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:18.853749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:18.915576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 584/1168] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:18.939003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc86de9-d919-45fd-9e36-0ac9913b3ad4 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:19.050368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc86de9-d919-45fd-9e36-0ac9913b3ad4 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:19.074517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:19.086175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:19.086829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:19.087141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:19.087414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:19.088153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:19.089668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50956e-486a-485e-8eb7-dffb9da10a8d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:19.110856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 585/1169] 10.222.0.74 () {60 vars in 1138 bytes} [Thu Jun 16 16:29:18 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:21.277882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234d46-d189-42c5-b8c6-717104bbb492 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:21.278443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234d46-d189-42c5-b8c6-717104bbb492 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:21.278443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93234d46-d189-42c5-b8c6-717104bbb492 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:21.279843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 585/1170] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:21.285537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:21.286338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:21.286497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:21.308079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:21.309339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:21.380018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 586/1171] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:21.404893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93234d46-d189-42c5-b8c6-717104bbb492 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:21.531940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93234d46-d189-42c5-b8c6-717104bbb492 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:21.581437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:21.582321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:21.582573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:21.582851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] PATH_INFO: `/v3/domains/dd131ed634d94fe1867086f5f401cfdf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:21.595694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd131ed634d94fe1867086f5f401cfdf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:21.597175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953b75a-818c-4402-8679-50ffe12b7774 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:21.598917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 586/1172] 10.222.0.74 () {56 vars in 1183 bytes} [Thu Jun 16 16:29:21 2022] GET /identity/v3/domains/dd131ed634d94fe1867086f5f401cfdf => generated 267 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:21.606579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:21.734957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676131c4-c441-4f15-9439-9dbe9f3dce0b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:21.753166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:21.785731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:21.786409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:21.786736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:21.787000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:21.787917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:21.789984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e75898-39ce-48af-8b03-a6581f23024f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:21.821279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 587/1173] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:29:21 2022] POST /identity/v3/projects => generated 361 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:24.085033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c8a7-75e1-4785-a21c-9dd710324213 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:24.085621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c8a7-75e1-4785-a21c-9dd710324213 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:24.085621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c8a7-75e1-4785-a21c-9dd710324213 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:24.086962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 587/1174] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:24.092280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:24.092697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:24.092805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:24.110753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:24.111986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:24.183038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 588/1175] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:24.207777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c8a7-75e1-4785-a21c-9dd710324213 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:24.221936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:24.320450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c8a7-75e1-4785-a21c-9dd710324213 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:24.355899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:24.356688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:24.356968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:24.357244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] PATH_INFO: `/v3/domains/dd131ed634d94fe1867086f5f401cfdf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:24.358464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd131ed634d94fe1867086f5f401cfdf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:24.359982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcacc0b-2eb9-4cf7-b7b1-d64f613468f1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:24.361622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 588/1176] 10.222.0.74 () {56 vars in 1183 bytes} [Thu Jun 16 16:29:24 2022] GET /identity/v3/domains/dd131ed634d94fe1867086f5f401cfdf => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:24.367978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:24.466978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:24.474138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afda9a-31b1-4ced-a447-19704314659e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:24.511978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:24.512655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:24.512943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:24.513263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:24.513985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:24.515455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa98767-6991-4fa6-8f2d-d05b58ea078b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:24.536643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 589/1177] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:29:24 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:26.685883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcd12-08cd-49d8-9a06-a88a225f49fc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:26.686431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcd12-08cd-49d8-9a06-a88a225f49fc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:26.686431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dcd12-08cd-49d8-9a06-a88a225f49fc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:26.687820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 589/1178] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:26.693142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:26.693564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:26.693672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:26.710862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:26.712087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:26.781918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 590/1179] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:26.805984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dcd12-08cd-49d8-9a06-a88a225f49fc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:26.887695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:26.911090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dcd12-08cd-49d8-9a06-a88a225f49fc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:26.946412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:26.947234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:26.947514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:26.947782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:26.958398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:26.965867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:26.966902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:29:26.967837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 590/1180] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:26.973456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:27.063496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:27.082826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1ab92-8d6a-4858-9369-8df39636e45a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:27.118203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:27.118866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:27.119175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:27.119458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:27.120200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:27.120508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:27.121863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:27.130449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 591/1181] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:27.136668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:27.242169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b98411-5bb2-46af-be83-7da42fc107b6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:27.278970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:27.279884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:27.280164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:27.280417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] PATH_INFO: `/v3/users/096c6105be9a44cfbd44437849fb916b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:27.288369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:27.294645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] RBAC: Authorizing `identity:get_user(user_id=096c6105be9a44cfbd44437849fb916b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:27.296132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:27.297790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 591/1182] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:27 2022] GET /identity/v3/users/096c6105be9a44cfbd44437849fb916b => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:27.304000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:27.410762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449b2f2-dda6-4566-9993-e757cdd541ba None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:27.450096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:27.450734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:27.451014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:27.451320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:27.468238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:27.526566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:27.528098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:27.636757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 592/1183] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:29:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:29:27.646153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:27.756130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759740a6-7809-4c0e-a50e-503b092e6636 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:27.792635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:27.793490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:27.793769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:27.794038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] PATH_INFO: `/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:27.795273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e0673f58ab7d4a66a9ad1a1dca2b7690)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:27.796692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19635545-87fa-4c01-a846-a9dbc41137c7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:27.798364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 592/1184] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:27 2022] GET /identity/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:27.805518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:27.862494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:27.919064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a5da7-b1ec-46fe-9bd3-54f4bea07a42 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:27.964518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:27.965170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:27.965465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:27.965745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:27.966583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=e0673f58ab7d4a66a9ad1a1dca2b7690, user.id=096c6105be9a44cfbd44437849fb916b) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:27.966894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:27.968380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978720d-f3b3-4c38-a3cf-9dc7957454f7 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:27.974036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 593/1185] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:29:27 2022] GET /identity/v3/role_assignments?user.id=096c6105be9a44cfbd44437849fb916b&scope.project.id=e0673f58ab7d4a66a9ad1a1dca2b7690&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:30.050541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d59b-f8ea-49f3-9eec-5451593304e5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.051190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d59b-f8ea-49f3-9eec-5451593304e5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.051190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d59b-f8ea-49f3-9eec-5451593304e5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.052494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 593/1186] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:30.059292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.060068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.060239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.081414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:30.082612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:30.156434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 594/1187] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:30.185869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512d59b-f8ea-49f3-9eec-5451593304e5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:30.264114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:30.311216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512d59b-f8ea-49f3-9eec-5451593304e5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:30.350643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:30.351550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.351831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.352102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.363712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:30.371343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:30.372491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:29:30.374112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 594/1188] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:30.381663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:30.433314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:30.495765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd0fcf-4b1c-4385-a743-7986862cc53b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:30.531502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:30.532168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.532408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.532720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.533480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:30.533755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:30.535215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:30.544032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 595/1189] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:30.550096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:30.655773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fdaf9-e966-436b-a9a3-a188a91b41ff None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:30.679123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:30.691729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:30.692605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.692885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.693163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] PATH_INFO: `/v3/users/096c6105be9a44cfbd44437849fb916b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.694551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] RBAC: Authorizing `identity:get_user(user_id=096c6105be9a44cfbd44437849fb916b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:30.696013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:30.697629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 595/1190] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:30 2022] GET /identity/v3/users/096c6105be9a44cfbd44437849fb916b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:30.706986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:30.835771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:30.839350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07166c4-a9a0-4440-8aff-7f99fc5567be None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:30.878264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:30.879119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:30.879425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:30.879691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] PATH_INFO: `/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:30.880909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] RBAC: Authorizing `identity:get_project(project_id=e0673f58ab7d4a66a9ad1a1dca2b7690)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:30.882482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6694f-10f8-46a9-bd6b-b18c2f1cf385 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:30.884230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 596/1191] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:30 2022] GET /identity/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690 => generated 361 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:30.893894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:30.998883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8934dc-4b96-4eed-b638-01a5f2a562a9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:31.036822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:31.037707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:31.038016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:31.038291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] PATH_INFO: `/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690/users/096c6105be9a44cfbd44437849fb916b/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:31.040452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] RBAC: Authorizing `identity:create_grant(project_id=e0673f58ab7d4a66a9ad1a1dca2b7690, user_id=096c6105be9a44cfbd44437849fb916b, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:31.041892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f269c5d-91b4-4d9c-8338-eb2b7618df3a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:31.058633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 596/1192] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:29:30 2022] PUT /identity/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690/users/096c6105be9a44cfbd44437849fb916b/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:33.236922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4ddd9-5be9-45a6-af66-54a1a293563f None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:33.237623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4ddd9-5be9-45a6-af66-54a1a293563f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:33.237623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4ddd9-5be9-45a6-af66-54a1a293563f None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:33.239179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 597/1193] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:29:33.245016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:33.245478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:33.245595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:33.267165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:33.269197 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:33.326588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:33.349289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 597/1194] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:29:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:29:33.373791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4ddd9-5be9-45a6-af66-54a1a293563f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:33.486816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4ddd9-5be9-45a6-af66-54a1a293563f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:33.525966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:33.526679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:33.526956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:33.527260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:33.540989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:29:33.548508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:29:33.549587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:29:33.550781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 598/1195] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:29:33.555948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:33.660046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b87c74-f244-4087-b899-2d365f6d9381 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:33.695084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:33.695762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:33.696038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:33.696311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:33.697046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:33.697320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:33.698719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:33.706767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 598/1196] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:33.713436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:33.745150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:33.820617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e758fb-a91c-49cc-926d-4828b8c12adc None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:33.858554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:33.859456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:33.859730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:33.860005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] PATH_INFO: `/v3/users/096c6105be9a44cfbd44437849fb916b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:33.861209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] RBAC: Authorizing `identity:get_user(user_id=096c6105be9a44cfbd44437849fb916b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:33.862830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:33.864477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 599/1197] 10.222.0.74 () {56 vars in 1177 bytes} [Thu Jun 16 16:29:33 2022] GET /identity/v3/users/096c6105be9a44cfbd44437849fb916b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:33.870583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:33.922569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:33.993027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5330ad-aed3-405d-bf55-404ce1918ac9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:34.033726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:34.034367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:34.034646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:34.034926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:34.104762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:34.106135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:34.154489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:34.216110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 599/1198] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:29:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:29:34.222410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:34.322319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:34.329621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5701-dc75-41d4-acd1-d8a8749d3620 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:34.363506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:34.364319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:34.364633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:34.364906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] PATH_INFO: `/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:34.366172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] RBAC: Authorizing `identity:get_project(project_id=e0673f58ab7d4a66a9ad1a1dca2b7690)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:34.367810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048582cd-b79e-46c9-95b1-316a43720080 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:34.369629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 600/1199] 10.222.0.74 () {56 vars in 1186 bytes} [Thu Jun 16 16:29:34 2022] GET /identity/v3/projects/e0673f58ab7d4a66a9ad1a1dca2b7690 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:34.379184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:34.478960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a339d-8031-46ef-8ba6-d6428b87b1a1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:34.517634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:34.518280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:34.518560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:34.518835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:34.519698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=e0673f58ab7d4a66a9ad1a1dca2b7690, user.id=096c6105be9a44cfbd44437849fb916b) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:34.520010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:34.521472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5173-0d17-4d8a-bb19-5b79e15243c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:34.527498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 600/1200] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:29:34 2022] GET /identity/v3/role_assignments?user.id=096c6105be9a44cfbd44437849fb916b&scope.project.id=e0673f58ab7d4a66a9ad1a1dca2b7690&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:29:44.359332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=18 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:44.524749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:33.232567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de760-efac-4f87-81c0-b33fcb374451 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:33.232567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de760-efac-4f87-81c0-b33fcb374451 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:33.232567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645de760-efac-4f87-81c0-b33fcb374451 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:33.232567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 601/1201] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:33.239467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:33.240104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:33.240382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:33.261262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:33.262478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:33.362764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 601/1202] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:30:33.390867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645de760-efac-4f87-81c0-b33fcb374451 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:33.531866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645de760-efac-4f87-81c0-b33fcb374451 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:33.602992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:33.603899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:33.604185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:33.604482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:33.617592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:33.618342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:33.639889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:33.641107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:33.642433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:33.642433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:33.642433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:33.642433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:33.642433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 602/1203] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:33.647806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:33.792438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e190e-4eb6-420a-aeaf-98c30802949d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:33.829697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:33.840247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:33.843212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:33.843212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:33.843212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:33.843212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:33.843212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:33.843862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4502e81-6af9-4e75-9642-62ffe17b27fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:33.859297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 602/1204] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:33.864674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:33.930733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:33.999891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf83572-4a12-446a-b0a8-5c5187cb7a37 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:34.047304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:34.047974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:34.048274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:34.048629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:34.049434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:34.052937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000f45-64ba-4c97-976c-e41e713113d9 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:34.081512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 603/1205] 10.222.0.74 () {60 vars in 1132 bytes} [Thu Jun 16 16:30:33 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:36.592399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf95f-3438-4e56-835d-4f213a31c04c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:36.592989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf95f-3438-4e56-835d-4f213a31c04c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:36.592989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf95f-3438-4e56-835d-4f213a31c04c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:36.594312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 603/1206] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:36.600439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:36.600851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:36.601003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:36.620237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:36.621443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:36.704812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 604/1207] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:30:36.733766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf95f-3438-4e56-835d-4f213a31c04c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:36.860621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf95f-3438-4e56-835d-4f213a31c04c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:36.903771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:36.904589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:36.904877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:36.905148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:36.916591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:30:36.926249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:36.927458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:36.928403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 604/1208] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:30:36.934307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:36.986490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:37.066624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10760381-27da-42a4-b681-936b83d32ef2 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:37.106397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:37.107306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:37.107602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:37.107958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:37.108873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:37.109263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:37.111276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:37.125621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 605/1209] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:37.131784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:37.208424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:37.267805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be884-5ce5-45bb-909d-4e4bd4cab57b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:37.307261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:37.308079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:37.308378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:37.308632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:37.320177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:37.323712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:37.341416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:37.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:37.343817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:37.343817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:37.343817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:37.343817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:37.344741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 605/1210] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:37.353452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:37.471812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:37.475527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7195-b9b1-45ef-bacd-c1bfd1410c3f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:37.520486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:37.521206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:37.521504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:37.521779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:37.522576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:37.522878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:37.524328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:37.541994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 606/1211] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:37.549449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:37.677950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e3680-e265-472f-957d-3c3d2ac553c0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:37.680314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:37.722705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:37.723468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:37.723749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:37.724019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:37.733453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:37.735061 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:30:37.753703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:37.755371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:37.756339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 606/1212] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:37.762831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:37.905987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066eb807-2023-4a7d-a492-c17db469492e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:37.950506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:37.951397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:37.951830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:37.952153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:37.953192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:37.953537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:37.954990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:37.982008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 607/1213] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:30:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:37.988001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:38.138272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405402-8819-4054-868a-bb2b3cd810b6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:38.181784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:38.182478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:38.182762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:38.183041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:38.226078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:38.283586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:38.285019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:38.435086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 607/1214] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:30:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:30:38.439100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:38.471940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:38.583966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd4f8-d51f-42ee-bce7-b5bdc697d04b None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:38.627583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:38.629079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:38.629555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:38.630026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:38.644086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:38.645638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:38.671826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:38.672828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:38.673798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:38.673798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:38.673798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:38.673798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:38.673798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 608/1215] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:38.679400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:38.756807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:38.805937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e3456-fdac-4dee-ba42-4d53fe142c1d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:38.847282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:38.848039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:38.848369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:38.848721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:38.849496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:38.849878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:38.851469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:38.865229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 608/1216] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:38.871481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:38.956337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:39.019239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e52b7a-a08d-435f-9c5f-925b51f6228e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.059442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.060183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.060466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.060740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.069543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.071325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:30:39.088020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:39.089034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:39.089948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:39.095472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:39.099972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 609/1217] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:39.231474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ea8f-f6fc-41d2-826b-9607a5b86096 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.257483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:39.273454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.275013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.275013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.275013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.275695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:39.276010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.277470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56a54f-f757-48d6-a8a2-91c63c114325 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:39.292863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 609/1218] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:39.300059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:39.461128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd307ef-8916-4653-a67b-492834f89583 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.515786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:39.523841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.524565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.525054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.525332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.526314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=9b84d7ac4698432e98a30fd39de251f0) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:39.526633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.528121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b0588-10d4-4b5e-8b53-28aa7f4b107d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:39.535886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 610/1219] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:30:39 2022] GET /identity/v3/role_assignments?user.id=9b84d7ac4698432e98a30fd39de251f0&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:41.965127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5391-51db-4ff3-9901-8e09f9e718fc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:41.965666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5391-51db-4ff3-9901-8e09f9e718fc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:41.965666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf5391-51db-4ff3-9901-8e09f9e718fc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:41.967244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 610/1220] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:41.972800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:41.973509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:41.973509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:41.994500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:41.996041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:42.072033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 611/1221] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:30:42.094672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf5391-51db-4ff3-9901-8e09f9e718fc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:42.232688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf5391-51db-4ff3-9901-8e09f9e718fc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:42.278055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:42.278822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:42.279134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:42.279414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:42.292718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:30:42.300712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:42.301686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:42.302501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 611/1222] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:30:42.308484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:42.429504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e1143-72ef-4dc9-a5ec-56ea235f5457 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:42.462116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:42.478671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:42.479386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:42.479666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:42.479942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:42.480700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] RBAC: Adding query filter params (name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:42.481004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:42.482379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:42.491732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 612/1223] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:42.497757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:42.572986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:42.658573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb1ef-d58d-4141-995d-3c2e54cd2ab1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:42.717466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:42.718340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:42.718586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:42.718861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:42.732097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:42.733856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:42.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:42.755535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:42.756848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:42.756848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:42.756848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:42.756848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:42.756848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 612/1224] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:42.762266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:42.890485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea703627-fcd6-4985-91ce-97ecc605a17d None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:42.935850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:42.936596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:42.936875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:42.937201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:42.937934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:42.938265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:42.940132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:42.962309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 613/1225] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:42.968673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:43.082072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:43.091963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ca43-d034-45c2-8cc6-8358346694da None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:43.135989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:43.136912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:43.137268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:43.137618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:43.147080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:43.148767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:30:43.166345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:43.167817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:43.168966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:43.171287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 613/1226] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:43.174510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:43.235744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:43.301505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1f441-fda5-41c7-8950-a8409aed3e73 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:43.349592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:43.350319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:43.350613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:43.350892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:43.351848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:43.352168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:43.353818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:43.373355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 614/1227] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:30:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:43.379348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:43.507453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071e58e-6b0e-43de-9512-2c6806a97118 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:43.545436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:43.546269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:43.546540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:43.546867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:43.559413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:43.560888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:43.578729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:43.579707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:43.580590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:43.580590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:43.580590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:43.580590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:43.580590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 614/1228] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:43.587753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:43.715311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d1f84-c475-481b-b894-25e79308537c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:43.718795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:43.760034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:43.760716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:43.760996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:43.761268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:43.761996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:43.762296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:43.763812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:43.784375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 615/1229] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:43.785707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:43.796400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:43.900229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed952e-6541-4c67-8727-c8cbaecc49e9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:43.952626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:43.953795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:43.954082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:43.954374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:43.965224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:43.967561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:30:43.985018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:43.986386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:43.987286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:43.991167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 615/1230] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:43.992177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:44.117004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ccb4-708a-49e1-b04b-a83e309e5000 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:44.159287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:44.160126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:44.160403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:44.160681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:44.161502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:44.161819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:44.163384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a0a-8d0a-4638-a003-c421fad96349 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:44.176784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 616/1231] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:30:44.183209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:44.262940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:44.302385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a45dd6-ba50-4615-a4c3-f91bb0758439 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:44.342117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:44.343289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:44.343783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:44.344305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/9b84d7ac4698432e98a30fd39de251f0/roles/86fa513c38c346c9a49dbc82991ff71a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:44.362088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=9b84d7ac4698432e98a30fd39de251f0, role_id=86fa513c38c346c9a49dbc82991ff71a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:44.363955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c2595-6516-4908-a074-d5bebe55c75c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:44.378762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 616/1232] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:30:44 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/9b84d7ac4698432e98a30fd39de251f0/roles/86fa513c38c346c9a49dbc82991ff71a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:30:46.638217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd186b-6b23-4b5a-97e0-3e0df33a03d6 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:46.638755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd186b-6b23-4b5a-97e0-3e0df33a03d6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:46.638755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd186b-6b23-4b5a-97e0-3e0df33a03d6 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:46.640362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 617/1233] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:46.645406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:46.645788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:46.645965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:46.663028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:46.664254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:46.738158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 617/1234] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:30:46.762622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd186b-6b23-4b5a-97e0-3e0df33a03d6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:46.789912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:46.873993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd186b-6b23-4b5a-97e0-3e0df33a03d6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:46.913313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:46.914439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:46.914894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:46.915197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:46.925942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:30:46.934253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:46.935241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:30:46.936096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:46.941233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 618/1235] 10.222.0.74 () {56 vars in 1102 bytes} [Thu Jun 16 16:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:46.941965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:46.999021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:47.067218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d490c2-1b22-47e6-9e94-688dc1ce9e8c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:47.114065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:47.114768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:47.115093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:47.117524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:47.117524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:47.117867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:47.119314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:47.130046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 618/1236] 10.222.0.74 () {56 vars in 1103 bytes} [Thu Jun 16 16:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:47.136140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:47.243696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:47.262678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180254b-ae5a-42cf-ab4e-191124af6a37 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:47.300474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:47.301345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:47.301733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:47.301985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:47.314302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:47.315781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:47.333473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:47.334660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:47.335715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:47.335715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:47.335715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:47.335715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:47.335715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 619/1237] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:47.340848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:47.469052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456164-f6d6-4a8f-812d-b7ac9c99876e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:47.478897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:47.511600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:47.512276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:47.512563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:47.512841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:47.513690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:47.514002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:47.515424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:47.534236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 619/1238] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:47.536380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:47.653528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654d0d6-64a2-4ed3-ba49-13842f5e0114 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:47.694220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:47.695325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:47.695692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:47.696350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:47.699707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:47.705683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:47.707506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:30:47.722145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:47.723509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:47.724764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:47.728584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 620/1239] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:47.729748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:47.850535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a2844-4bf5-42c3-b94e-cc88e026074d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:47.896192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:47.896880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:47.897161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:47.897457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:47.898278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:47.898587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:47.900350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:47.918987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 620/1240] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:30:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:47.925055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.038448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9243d-d0ce-484d-b077-eac3e711ffb5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.076960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.077761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.078057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.078356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.145321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.152045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.153517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.274203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 621/1241] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:30:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:30:48.280202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.310911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.391327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46dd45-8bc1-419f-be18-8c8f061c7f3e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.433077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.433953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.434240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.434568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.448196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.450129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:48.472104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:48.473277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:48.474569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:48.474569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:48.474569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:48.474569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:48.474569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 621/1242] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:48.479876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.592029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.598993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310e3d-705a-4adf-b146-389dc2dcde8c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.635628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.636513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.636909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.637225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.637967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:48.638424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.640199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.654874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 622/1243] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:48.661042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.742078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.764356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138661ee-7ece-48c9-8411-58f9ad2c5f9f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.800650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.801455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.801734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.802003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.809144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.810596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:30:48.825348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:48.826259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:48.827419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 622/1244] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:30:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:30:48.832790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.952318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e8820-cda8-42cd-9ebc-a942e3ea9b01 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.989628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.990433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.990719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.991172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.992060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:48.992403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.994259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05600bd6-ff86-4acb-95c0-0fde19cc3b78 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:49.010445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 623/1245] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:49.016818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:49.152121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9258b-990c-416b-a589-c7e3056306ba None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:49.196879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:49.197562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:49.197838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:49.198216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:49.198957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] RBAC: Adding query filter params (role.id=86fa513c38c346c9a49dbc82991ff71a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=9b84d7ac4698432e98a30fd39de251f0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:49.199501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:49.200771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed32228-3d2a-4f70-acad-e96ee9edebe9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:49.209056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 623/1246] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:30:49 2022] GET /identity/v3/role_assignments?user.id=9b84d7ac4698432e98a30fd39de251f0&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=86fa513c38c346c9a49dbc82991ff71a => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:30:51.465777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e83b0-5c0c-41b2-92da-73d93e9626a5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:51.466370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e83b0-5c0c-41b2-92da-73d93e9626a5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:51.466628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e83b0-5c0c-41b2-92da-73d93e9626a5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:51.468277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 624/1247] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:51.475175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:51.475617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:51.475977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:51.483487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:51.498714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:51.499972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:51.585337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 624/1248] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:30:51.610295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e83b0-5c0c-41b2-92da-73d93e9626a5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:51.631844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:51.731135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e83b0-5c0c-41b2-92da-73d93e9626a5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:51.769289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:51.770078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:51.770360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:51.770624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:51.781997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:30:51.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:51.791581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:51.792422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 625/1249] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:51.797821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:51.902912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302e775-22ab-4561-bf92-9f1426ca129c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:51.934214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:51.939565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:51.940250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:51.940530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:51.940805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:51.941549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:51.941858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:51.943338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:51.954257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 625/1250] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:51.958860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:52.045531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:52.075272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64a46a-1237-4b2b-8bc8-f4090fc00f59 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:52.115589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:52.116542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:52.116913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:52.117359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:52.129448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:52.130920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:52.148534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:52.149611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:52.150482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:52.150482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:52.150482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:52.150482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:52.150710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 626/1251] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:52.156483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:52.271356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb329c8-775b-46ac-9ffe-7a7b2c28e7d6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:52.310262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:52.310861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:52.311163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:52.311441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:52.312162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:52.312455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:52.313831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:52.329139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 626/1252] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:52.338522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:52.452819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34cfdf-d833-4d4c-9d4e-472bd54f3703 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:52.482526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:52.491537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:52.492360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:52.492672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:52.493083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:52.502081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:52.503556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:30:52.518218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:52.519424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:52.520398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 627/1253] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:52.526300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:52.586423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:52.635336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49b9f9-dc2f-4f82-a90d-563859d62180 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:52.671695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:52.672394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:52.672683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:52.672960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:52.673842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:52.674149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:52.675798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:52.694170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 627/1254] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:30:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:52.701247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:52.814462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bdd2c-0d35-4bc4-9c9d-b4888bdfeb7c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:52.851997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:52.852540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:52.853001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:52.853093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:52.913854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:52.928980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:52.931483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.041306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 628/1255] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:30:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:30:53.047412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:53.157458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb7944-5c44-49a6-827e-5b84ad542848 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:53.175362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:53.203384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:53.204807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:53.205116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:53.205534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:53.220556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:53.222559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:53.246644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:53.247696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:53.248561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:53.248561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:53.248561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:53.248561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:53.248561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 628/1256] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:30:53.254802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:53.355529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:53.370434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09633-708b-4b19-b970-f7e51d5cf77f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:53.409118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:53.409822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:53.410104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:53.410378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:53.411292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:53.411685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:53.413273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.427145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 629/1257] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:30:53.433865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:53.551072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75485-ac19-45fa-a0d0-152fe1aa8f60 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:53.588483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:53.589425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:53.589766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:53.590040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:53.597524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:53.598985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:30:53.614285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:53.615343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:30:53.616280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 629/1258] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:53.622302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:53.729359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554da4-424c-415a-bb19-cb44cd56958c None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:53.767973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:53.768642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:53.768925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:53.769202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:53.769990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:53.770302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:53.771765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6be46a-fc1d-49a7-8b6e-16389e90fe57 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.786518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 630/1259] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:53.792337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:53.819901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:53.902272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a64d-1b35-4219-ba9e-85ad73d8e935 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:53.942071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:53.942759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:53.943083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:53.943353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:53.944213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=9b84d7ac4698432e98a30fd39de251f0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:53.944530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:53.946083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690386a-c5b9-4309-b113-43feb23ac6a1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:53.952524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 630/1260] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:30:53 2022] GET /identity/v3/role_assignments?user.id=9b84d7ac4698432e98a30fd39de251f0&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:30:56.063216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc3b-5dea-4764-9f31-9ecd5dc4f590 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.064065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc3b-5dea-4764-9f31-9ecd5dc4f590 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.064065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc3b-5dea-4764-9f31-9ecd5dc4f590 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.066281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 631/1261] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:30:56.072566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.073328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.073505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.101038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:56.102691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:56.188385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 631/1262] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:30:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:30:56.216529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12dc3b-5dea-4764-9f31-9ecd5dc4f590 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:56.228173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:56.339599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12dc3b-5dea-4764-9f31-9ecd5dc4f590 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:56.375634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:56.376424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.376705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.377288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.388769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:30:56.396362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:56.397463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:56.398335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 632/1263] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:56.404542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:56.427880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:56.516862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0d86f-d2dc-4d89-9031-918bb00e08c7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:56.552352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:56.553029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.553313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.553590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.554343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:56.554651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:56.556143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:56.564636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 632/1264] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:56.570555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:56.641298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:56.675501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f3158-af2c-4170-af86-eea99188b277 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:56.714803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:56.715625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.715908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.716180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.727218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:56.728668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:56.745914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:56.746952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:56.747892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:56.747892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:56.747892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:56.747892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:56.747892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 633/1265] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:56.755628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:56.841857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:56.866644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20e95a-f626-4cdb-bc4f-6a2431d1a5fa None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:56.902243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:56.902831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:56.903111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:56.903410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:56.904120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:56.904426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:56.905796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:56.918573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 633/1266] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:56.925133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.030143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0763fa-cc2b-4323-93ac-68f5f797e19a None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.050161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:57.065138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.065937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.066214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.066474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.073982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.075449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:30:57.089267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:57.090322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:57.091341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 634/1267] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:57.096890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.221524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fe41-7a3d-4f1a-84c8-ae68edd71b6a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.257197 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.257864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.258147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.258485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.259490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:57.259805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.261255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.270094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:57.280621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 634/1268] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:30:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:57.286577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.386759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd1eef-6e29-4256-bae1-f7b4fa121af8 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.421230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.422009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.422284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.422550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.434199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.435666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:30:57.453584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:57.454600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:57.455668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:30:57.455668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:30:57.455668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:30:57.455668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:57.455668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 635/1269] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:57.460669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.564914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a73d6c-01f9-4d1b-b607-486635359809 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.602889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.603589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.603880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.604152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.604885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:57.605186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.606591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.618070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 635/1270] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:57.623903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.660104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:57.735954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2157-1ecf-4d5a-8e28-37ac45aa7fe2 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.780355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.781148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.781433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.781707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.788921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.790413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:30:57.804526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:30:57.805427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:30:57.806700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 636/1271] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:30:57.812621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:57.874676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:57.917406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6aa456-2fc1-4505-8f3b-775eb4525430 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:57.953163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:57.953820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:57.954101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:57.954378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:57.955241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:57.955555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:57.956995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222140a-9c0d-42c0-967a-a6d685f2fba3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:57.969163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 636/1272] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:57.978698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:58.084945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:58.094972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e703ba-5c86-469d-800a-fe8dba89a04e None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:58.131829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:58.132730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:58.133012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:58.133290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] PATH_INFO: `/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/9b84d7ac4698432e98a30fd39de251f0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:58.135945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] RBAC: Authorizing `identity:create_grant(project_id=c245ad526c074bffaaa04e41f9f74e55, user_id=9b84d7ac4698432e98a30fd39de251f0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:58.137386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45270ba-5ba3-444a-b4eb-1903797a4467 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:58.154557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 637/1273] 10.222.0.74 () {58 vars in 1439 bytes} [Thu Jun 16 16:30:57 2022] PUT /identity/v3/projects/c245ad526c074bffaaa04e41f9f74e55/users/9b84d7ac4698432e98a30fd39de251f0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:31:00.322739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a40c8-337a-4084-bdcb-b594fdfaeea6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.323585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a40c8-337a-4084-bdcb-b594fdfaeea6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.323585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a40c8-337a-4084-bdcb-b594fdfaeea6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.325112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 637/1274] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:31:00.332057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.332504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.332614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.350704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:00.351930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:00.432207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 638/1275] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:31:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:31:00.466859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a40c8-337a-4084-bdcb-b594fdfaeea6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:00.583865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a40c8-337a-4084-bdcb-b594fdfaeea6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:00.603508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:00.621937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:00.622744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.623035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.623343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.635357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:31:00.642985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:00.643932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:00.644776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 638/1276] 10.222.0.74 () {56 vars in 1096 bytes} [Thu Jun 16 16:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:00.650990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:00.739866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:00.759538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704894f9-e1fb-41f4-8900-d470811484c5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:00.806138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:00.806807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.807101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.807385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.808134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:00.808435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:00.809812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:00.818913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 639/1277] 10.222.0.74 () {56 vars in 1099 bytes} [Thu Jun 16 16:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:00.825810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:00.952084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287b8e0-4218-4b56-9d2b-77f81302ea8c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:00.989436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:00.990234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.990523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.990792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:01.005555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:01.007022 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:31:01.026860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:01.028143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:01.029440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:31:01.029440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:31:01.029440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:31:01.029440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:01.029440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 639/1278] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:01.034463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:01.148881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd78957-cea0-4771-8b13-fbeff7acfa95 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:01.189765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:01.190639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:01.190949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:01.191476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:01.192505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:01.192835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:01.194513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:01.213173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 640/1279] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:01.219152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:01.257676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:01.333719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f17a6-b5a8-4681-9b97-782b2f9e7d1c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:01.370415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:01.371256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:01.371539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:01.371809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:01.381180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:01.383132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:31:01.398359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:01.399492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:01.400526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 640/1280] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:31:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:01.405970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:01.410052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:01.511834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f7f0-5298-4f78-8e79-eadf076ff9e5 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:01.552414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:01.553099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:01.553380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:01.553652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:01.554504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:01.554810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:01.556710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:01.574798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 641/1281] 10.222.0.74 () {56 vars in 1143 bytes} [Thu Jun 16 16:31:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:01.580997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:01.676891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:01.687855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff9123-b22c-432f-a570-8ecf2d75d7ea None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:01.727764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:01.728506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:01.728788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:01.729062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:01.802214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:01.803707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:01.903780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:01.915021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 641/1282] 10.222.0.74 () {58 vars in 1282 bytes} [Thu Jun 16 16:31:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:31:01.921653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:01.989846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:02.033924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b86719-85f3-41cd-8d1b-255d3f1925e0 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:02.072927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:02.073640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:02.073928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:02.074207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:02.085043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:02.086501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:31:02.106031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:02.107150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:02.108041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:31:02.108041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:31:02.108041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:31:02.108041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:31:02.108041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 642/1283] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:02.113298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:02.221247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd071a-69c3-461c-abc6-51a80cce2780 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:02.263837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:02.264498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:02.264782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:02.265040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:02.265770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:02.266077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:02.267494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:02.279894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 642/1284] 10.222.0.74 () {56 vars in 1109 bytes} [Thu Jun 16 16:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:02.285961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:02.400884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b04ce1-a955-4e97-8985-7a9951f5bbeb None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:02.416797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:02.440014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:02.440821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:02.441105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:02.441386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:02.448842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:02.450289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:31:02.464673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:02.465677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:31:02.466735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: ERROR keystone.server.flask.application Jun 16 16:31:02.470840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 643/1285] 10.222.0.74 () {56 vars in 1111 bytes} [Thu Jun 16 16:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:02.471818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:02.529269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:02.589042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463118-5398-433f-8b1d-41bb9450510d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:02.625147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:02.625823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:02.626107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:02.626379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:02.627196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:02.627819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:02.629271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd143165-f0c8-4892-88a9-4f13fe82e201 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:02.642236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 643/1286] 10.222.0.74 () {56 vars in 1148 bytes} [Thu Jun 16 16:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:02.648563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:02.757364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c07eed-686c-43ef-8443-c37e714d5639 None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:02.795659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:02.796321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:02.796603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:02.796881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:02.797716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] RBAC: Adding query filter params (role.id=279eea356bbd45f39595ccaa1656385a, scope.project.id=c245ad526c074bffaaa04e41f9f74e55, user.id=9b84d7ac4698432e98a30fd39de251f0) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:02.798031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:02.799499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7227a-660c-4f1f-adbf-136ff6bb38fd None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:02.805839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 644/1287] 10.222.0.74 () {56 vars in 1374 bytes} [Thu Jun 16 16:31:02 2022] GET /identity/v3/role_assignments?user.id=9b84d7ac4698432e98a30fd39de251f0&scope.project.id=c245ad526c074bffaaa04e41f9f74e55&role.id=279eea356bbd45f39595ccaa1656385a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:04.921339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbe008-5e78-4b10-bc74-9859feafbeff None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:04.921861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbe008-5e78-4b10-bc74-9859feafbeff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:04.921861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbe008-5e78-4b10-bc74-9859feafbeff None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:04.923939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 644/1288] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:31:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:31:04.932610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:04.933024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:04.933239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:04.951716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:04.952881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:04.986605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:05.029475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 645/1289] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:31:05.051874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbe008-5e78-4b10-bc74-9859feafbeff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:05.168342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbe008-5e78-4b10-bc74-9859feafbeff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:05.182623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:05.213232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:05.214044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:05.214326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:05.214598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:05.215282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:05.216669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:05.229293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:05.230376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:05.231443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 645/1290] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:31:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:05.236743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:05.349234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5797e2-b46e-430f-9fe5-a54efe29c95f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:05.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:05.385455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:05.385730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:05.386002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:05.386708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:05.387011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:05.388431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acba8a-cbbb-473b-8d2a-8fa021ebfa62 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:05.393501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 646/1291] 10.222.0.74 () {56 vars in 1116 bytes} [Thu Jun 16 16:31:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:05.399803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:05.506216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496a50a-201a-4b7b-b8e4-1e24157b7b59 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:05.542965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:05.543674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:05.543954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:05.544226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:05.544913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:05.545220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:05.546589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88b9ff-929c-436a-b617-27db043bf9e0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:05.551745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 646/1292] 10.222.0.74 () {56 vars in 1116 bytes} [Thu Jun 16 16:31:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:07.633133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5148da9-de01-4980-8376-6a4f20054936 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:07.635113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5148da9-de01-4980-8376-6a4f20054936 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:07.635113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5148da9-de01-4980-8376-6a4f20054936 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:07.636276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 647/1293] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:31:07.642467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6f899-c972-451d-9550-49a29ec2a26b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:07.643258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6f899-c972-451d-9550-49a29ec2a26b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:07.643430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6f899-c972-451d-9550-49a29ec2a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:07.669474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2ed6f899-c972-451d-9550-49a29ec2a26b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:07.670696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2ed6f899-c972-451d-9550-49a29ec2a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:07.702767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:07.754449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 647/1294] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:31:07.780866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5148da9-de01-4980-8376-6a4f20054936 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:07.816632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:07.898098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5148da9-de01-4980-8376-6a4f20054936 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:07.933518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:07.934164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:07.934441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:07.934713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:07.935462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:07.936822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74233d2f-97f7-4697-a856-49b48bf70564 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:07.953336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 648/1295] 10.222.0.74 () {60 vars in 1141 bytes} [Thu Jun 16 16:31:07 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:31:10.068212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d393-7b2b-4e44-b3c8-3a3c8f47b87f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.068799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d393-7b2b-4e44-b3c8-3a3c8f47b87f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.068799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d393-7b2b-4e44-b3c8-3a3c8f47b87f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.070204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 648/1296] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:31:10.076528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.077254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.077428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.100146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:10.101354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:10.183384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 649/1297] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:31:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:31:10.217539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93d393-7b2b-4e44-b3c8-3a3c8f47b87f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:10.332245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93d393-7b2b-4e44-b3c8-3a3c8f47b87f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:10.367834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:10.368683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.368965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.369241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.369895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:10.371305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:10.383229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:10.384796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:10.385831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 649/1298] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:31:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:10.392439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:10.421582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:10.510159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bcb5b-310b-4db9-a9d9-9bacc6eb4de8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:10.548507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:10.548945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.549233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.549512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.550226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:10.550534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:10.551966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:10.558149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 650/1299] 10.222.0.74 () {56 vars in 1116 bytes} [Thu Jun 16 16:31:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:10.564252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:10.598803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:10.676301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ab88-12bd-4431-b638-116081650113 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:10.712928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:10.713584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.713870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.714146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.714935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] RBAC: Adding query filter params (interface=public, service_id=9a5373a0f87f438fa1fffe29074c98ad, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:10.715281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:10.716672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3c51e-ebca-4860-8a5c-dec993455b4f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:10.721837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 650/1300] 10.222.0.74 () {56 vars in 1251 bytes} [Thu Jun 16 16:31:10 2022] GET /identity/v3/endpoints?service_id=9a5373a0f87f438fa1fffe29074c98ad&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:10.728473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:10.822617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:10.845496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eeccb8-700d-4bee-bdaa-1fb5612c7d7f None admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:10.887387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:10.888286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:10.888833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:10.889228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:10.890290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:10.892491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9f173-269e-47ca-9408-8cfb9fd43bdb None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:10.902464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 651/1301] 10.222.0.74 () {56 vars in 1087 bytes} [Thu Jun 16 16:31:10 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:31:13.023821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d17fe-2167-4261-9b87-97c7731ec5ae None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:13.024437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d17fe-2167-4261-9b87-97c7731ec5ae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:13.024437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d17fe-2167-4261-9b87-97c7731ec5ae None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:13.026617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 651/1302] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:31:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:31:13.034533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:13.034977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:13.035108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:13.053335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:13.054570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:13.130787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 652/1303] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:31:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:31:13.153791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d17fe-2167-4261-9b87-97c7731ec5ae None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:13.258291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:13.266105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d17fe-2167-4261-9b87-97c7731ec5ae None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:13.304701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:13.305420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:13.305778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:13.306057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:13.306707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:13.308106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:31:13.320165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:31:13.321298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jun 16 16:31:13.322455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 652/1304] 10.222.0.74 () {56 vars in 1117 bytes} [Thu Jun 16 16:31:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:31:13.330808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:13.413627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:13.445220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cb2db-50a3-4413-81b3-d64f3f49f61d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:13.484418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:13.485080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:13.485364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:13.485642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] PATH_INFO: `/v3/services` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:13.486347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:31:13.486655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:13.488064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2be281-0c13-4fc1-a71f-25a7364f3528 None admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:13.493380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 653/1305] 10.222.0.74 () {56 vars in 1116 bytes} [Thu Jun 16 16:31:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:31:13.502076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:13.614317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71620c-1d90-4128-b6ef-4b6be4c998f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:13.652739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:13.653391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:13.653676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:13.653947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:13.654681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:13.656086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a7eb0-1633-452e-a744-db0bf124dd88 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:13.678358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 653/1306] 10.222.0.74 () {60 vars in 1144 bytes} [Thu Jun 16 16:31:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:31:23.490594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=30 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:23.675036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:17.363448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1295149-2faf-4177-8cce-484cb8357229 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:17.364051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1295149-2faf-4177-8cce-484cb8357229 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:17.364051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1295149-2faf-4177-8cce-484cb8357229 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:17.370191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 654/1307] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:17.372485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87e01-ab9c-4a38-8aef-5b7184303d2f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:17.373045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87e01-ab9c-4a38-8aef-5b7184303d2f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:17.373045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87e01-ab9c-4a38-8aef-5b7184303d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:17.477530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b3c87e01-ab9c-4a38-8aef-5b7184303d2f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:17.478752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b3c87e01-ab9c-4a38-8aef-5b7184303d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:17.598868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 654/1308] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:32:17.656168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:17.656380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:17.656764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:17.658382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 655/1309] 10.222.0.74 () {54 vars in 964 bytes} [Thu Jun 16 16:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:17.664902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fd26-5f91-45fc-96a3-6077f6a0fb11 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:17.665419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fd26-5f91-45fc-96a3-6077f6a0fb11 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:17.666116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fd26-5f91-45fc-96a3-6077f6a0fb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:17.756470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7d17fd26-5f91-45fc-96a3-6077f6a0fb11 None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:17.758333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7d17fd26-5f91-45fc-96a3-6077f6a0fb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:17.784122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:17.864397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:17.866151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:17.875414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 655/1310] 10.222.0.74 () {58 vars in 1061 bytes} [Thu Jun 16 16:32:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:18.020099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eabd2-b636-463b-bfe5-a0ff37a55ebe None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:18.109292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:18.109958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:18.110244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:18.110524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:18.112401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:18.188336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:18.257442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:18.259034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050d87a-dd77-4692-a56b-3993fffb07ae service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:18.294069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 656/1311] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:32:20.748108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:20.748698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:20.748698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:20.750049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 656/1312] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:20.755324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2eff7-4fdd-4f25-8009-3021f0339525 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:20.755752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2eff7-4fdd-4f25-8009-3021f0339525 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:20.755866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2eff7-4fdd-4f25-8009-3021f0339525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:20.844098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5ba2eff7-4fdd-4f25-8009-3021f0339525 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:20.845204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5ba2eff7-4fdd-4f25-8009-3021f0339525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:20.944618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 657/1313] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:20.982347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:20.983959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:21.092945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c60950-5ace-4056-b1ed-414e7790075c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:21.168981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:21.172004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:21.172753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:21.173042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:21.173309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:21.175651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:21.307782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:21.309380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1a1c0-f3cc-451b-82d7-0a12821f5d55 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:21.330498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 657/1314] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:23.759640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:23.760526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:23.760526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:23.762742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 658/1315] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:23.768195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882dfd-4103-4617-a417-39165e0eedd3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:23.768639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882dfd-4103-4617-a417-39165e0eedd3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:23.768734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882dfd-4103-4617-a417-39165e0eedd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:23.834379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7c882dfd-4103-4617-a417-39165e0eedd3 None None] MFA Rules not processed for user `f7faeca25d31483582cb85013fa9b8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:23.835559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7c882dfd-4103-4617-a417-39165e0eedd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:23.889111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:23.937561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 658/1316] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:23.979955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:23.981524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:23.992409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:24.100272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942129d-6720-4581-bc0f-b6f850a918ad None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:24.181011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:24.181655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:24.181939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:24.182210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:24.184063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:24.266914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:24.321147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:24.322589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74192d9-a303-4fc5-b1c9-de683979492c service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:24.340704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 659/1317] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:27.086305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:27.087245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:27.087245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:27.089317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 659/1318] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:27.097720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2281b-934a-435b-97d8-23c79dc4c957 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:27.098406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2281b-934a-435b-97d8-23c79dc4c957 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:27.098595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2281b-934a-435b-97d8-23c79dc4c957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:27.177237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1ba2281b-934a-435b-97d8-23c79dc4c957 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:27.178540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1ba2281b-934a-435b-97d8-23c79dc4c957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:27.286314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 660/1319] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:27.328214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:27.329753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:27.449124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa72858-2182-48cf-a85f-4f0050fef1ac None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:27.536255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:27.536915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:27.537198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:27.537469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:27.539180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:27.547635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:27.673349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:27.674802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836edca-64a2-4d69-aefc-7ae52451de6c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:27.695585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 660/1320] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:30.476965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:30.477528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:30.477528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:30.478880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 661/1321] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:30.485198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a99f35-f2ae-47a5-bfb6-3995eb6c6980 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:30.485652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a99f35-f2ae-47a5-bfb6-3995eb6c6980 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:30.486301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a99f35-f2ae-47a5-bfb6-3995eb6c6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:30.557596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-06a99f35-f2ae-47a5-bfb6-3995eb6c6980 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:30.559267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-06a99f35-f2ae-47a5-bfb6-3995eb6c6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:30.626629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:30.672149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 661/1322] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:30.709450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:30.711164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:30.733824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:30.827224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dd73b-71d4-49a1-85f6-ed17bcf764dc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:30.902981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:30.903693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:30.903976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:30.904254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:30.905895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:30.983795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:31.024194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:31.025615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf63f1-73a6-4e2c-b432-d44d5e096f4c service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:31.047239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 662/1323] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:33.241818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33698-f438-4600-8b24-8e1d9ba954d7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:33.242529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33698-f438-4600-8b24-8e1d9ba954d7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:33.242529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33698-f438-4600-8b24-8e1d9ba954d7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:33.243759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 662/1324] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:33.249224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74323d73-bd6c-481f-9b55-e54d2d3c83d1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:33.249704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74323d73-bd6c-481f-9b55-e54d2d3c83d1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:33.249814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74323d73-bd6c-481f-9b55-e54d2d3c83d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:33.318664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-74323d73-bd6c-481f-9b55-e54d2d3c83d1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:33.320565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-74323d73-bd6c-481f-9b55-e54d2d3c83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:33.425949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 663/1325] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:33.473472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:33.473763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:33.474043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:33.475503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 663/1326] 10.222.0.74 () {54 vars in 964 bytes} [Thu Jun 16 16:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:33.481646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ebe37-a9fb-4184-b881-b3d1c098f018 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:33.481912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ebe37-a9fb-4184-b881-b3d1c098f018 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:33.482186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ebe37-a9fb-4184-b881-b3d1c098f018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:33.514495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:33.570983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-3c0ebe37-a9fb-4184-b881-b3d1c098f018 None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:33.572251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3c0ebe37-a9fb-4184-b881-b3d1c098f018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:33.653972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 664/1327] 10.222.0.74 () {58 vars in 1061 bytes} [Thu Jun 16 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:33.662972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:33.664539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:33.782665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1554e8-5439-4309-9337-8b94e0e8010f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:33.857786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:33.858441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:33.858719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:33.858992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:33.860659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:33.878715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:33.977686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:33.979179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deb731-5c42-41b0-b386-89994c02840b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:33.998055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 664/1328] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:36.453563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:36.454093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:36.454188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:36.455585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 665/1329] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:36.461642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed48e-3fa9-46b3-b904-e475d3dbf359 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:36.462124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed48e-3fa9-46b3-b904-e475d3dbf359 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:36.462218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aed48e-3fa9-46b3-b904-e475d3dbf359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:36.534531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f6aed48e-3fa9-46b3-b904-e475d3dbf359 None None] MFA Rules not processed for user `f7faeca25d31483582cb85013fa9b8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:36.536630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f6aed48e-3fa9-46b3-b904-e475d3dbf359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:36.611665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:36.645436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 665/1330] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:36.682764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:36.684464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:36.800243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f7883-ad28-4d97-bb5f-768abe1fcf9e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:36.826711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:36.890721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:36.891425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:36.891714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:36.891988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:36.893686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:37.016026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:37.017488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467013f4-476b-44f7-a04b-f36f0bf21114 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:37.038472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 666/1331] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:36 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:39.526731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:39.527669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:39.527669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:39.529414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 666/1332] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:39.537997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2c8b2-5e13-4689-973a-0c62e1bf37cf None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:39.538557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2c8b2-5e13-4689-973a-0c62e1bf37cf None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:39.538557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2c8b2-5e13-4689-973a-0c62e1bf37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:39.612888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:39.616283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9fd2c8b2-5e13-4689-973a-0c62e1bf37cf None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:39.618625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9fd2c8b2-5e13-4689-973a-0c62e1bf37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:39.724306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 667/1333] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:39.759504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:39.761067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:39.869481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0009a1f-20dd-4f16-b01a-e9bb385b2afc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:39.949897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:39.950556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:39.950837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:39.951201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:39.952872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:39.980342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:40.073882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:40.075854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa05eb-188b-4efe-8524-c13d271339cd service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:40.100138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 667/1334] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:42.252924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:42.253560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:42.253560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:42.255174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 668/1335] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:42.261807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8202-b896-40e0-8939-483142abbb34 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:42.262368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8202-b896-40e0-8939-483142abbb34 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:42.262368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8202-b896-40e0-8939-483142abbb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:42.340094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-313f8202-b896-40e0-8939-483142abbb34 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:42.341333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-313f8202-b896-40e0-8939-483142abbb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:42.434740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:42.459329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 668/1336] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:42.504118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:42.506522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:42.628320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5d93e-3abd-4178-a224-885b74f422a5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:42.658043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:42.711209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:42.711859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:42.712148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:42.712422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:42.714112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:42.834867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:42.836308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbf011-502b-424f-b59d-9ab1fe12bb5e service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:42.856797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 669/1337] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:45.615643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:45.616342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:45.616342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:45.618388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 669/1338] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:45.624782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473674d5-d0cc-46d6-b15a-e5cef37da205 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:45.625246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473674d5-d0cc-46d6-b15a-e5cef37da205 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:45.625354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473674d5-d0cc-46d6-b15a-e5cef37da205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:45.701536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-473674d5-d0cc-46d6-b15a-e5cef37da205 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:45.702747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-473674d5-d0cc-46d6-b15a-e5cef37da205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:45.711709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:45.811822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 670/1339] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:45.852305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:45.854776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:45.976763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565d49-27af-460c-a806-ec9ab576b986 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:46.064839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:46.065416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:46.065695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:46.065964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:46.067662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:46.108591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:46.187367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:46.188827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10ea6-ecec-4492-bd1d-9a11d8a5b735 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:46.208037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 670/1340] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:50.496170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:50.496758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:50.496758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:50.498411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 671/1341] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:50.503747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c31bb-b6e4-4560-ba5a-d8015fcc60c1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:50.504207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c31bb-b6e4-4560-ba5a-d8015fcc60c1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:50.504449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c31bb-b6e4-4560-ba5a-d8015fcc60c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:50.586699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-256c31bb-b6e4-4560-ba5a-d8015fcc60c1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:50.587873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-256c31bb-b6e4-4560-ba5a-d8015fcc60c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:50.683879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:50.697745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 671/1342] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:50.736363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:50.738767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:50.839882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ffec9-efcd-443b-8c76-a732857cb69e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:50.843540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:50.844286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:50.844719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:50.844980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:50.857213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:50.858676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8243e41-5f5d-4fc0-a11c-0c0ac5ea0c5d service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:50.869346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 672/1343] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:53.676764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:53.677387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:53.677387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:53.678759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 672/1344] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:53.684965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bad17-6967-475b-86da-cfde5caabee5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:53.685564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bad17-6967-475b-86da-cfde5caabee5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:53.685564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bad17-6967-475b-86da-cfde5caabee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:53.715205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-781bad17-6967-475b-86da-cfde5caabee5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:53.716373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-781bad17-6967-475b-86da-cfde5caabee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:53.748903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:32:53.805218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 673/1345] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:53.842747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:53.844611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:53.858599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7504677-59bd-41c6-8f7e-0035dcac403b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:53.862248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:53.862905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:53.863230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:53.863512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:53.873248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:53.874693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225d58b-f272-4d8e-b36f-43be36fa611c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:53.884571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 673/1346] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:32:58.202021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c128f2-026f-4064-8bea-386c58de220a None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:58.202594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c128f2-026f-4064-8bea-386c58de220a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:58.202594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c128f2-026f-4064-8bea-386c58de220a None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:58.204009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 674/1347] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:32:58.209322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872168-66ad-43ba-b73a-52d7a7e3005b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:58.209734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872168-66ad-43ba-b73a-52d7a7e3005b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:58.209879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872168-66ad-43ba-b73a-52d7a7e3005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:58.227452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e1872168-66ad-43ba-b73a-52d7a7e3005b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:32:58.228657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e1872168-66ad-43ba-b73a-52d7a7e3005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:32:58.238179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 674/1348] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:32:58.275879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c128f2-026f-4064-8bea-386c58de220a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:32:58.287600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c128f2-026f-4064-8bea-386c58de220a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:32:58.291286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:32:58.291932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:32:58.292221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:32:58.292501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:32:58.302727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:32:58.304216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797061-be9a-42f5-81c4-3d92172d3215 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:32:58.314591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 675/1349] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:32:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:03.348220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99b2d-412e-40cd-87ab-93d322d07e19 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:03.348792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99b2d-412e-40cd-87ab-93d322d07e19 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:03.349058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99b2d-412e-40cd-87ab-93d322d07e19 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:03.354954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 675/1350] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:03.357338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74d0b-b57e-4e4e-90aa-c8aaee39cd2f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:03.357923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74d0b-b57e-4e4e-90aa-c8aaee39cd2f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:03.358279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74d0b-b57e-4e4e-90aa-c8aaee39cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:03.438328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8dc74d0b-b57e-4e4e-90aa-c8aaee39cd2f None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:03.439959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8dc74d0b-b57e-4e4e-90aa-c8aaee39cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:03.520154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 676/1351] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:33:07.759600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d3d2-0bac-4382-977f-2ad2aeb8e28a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:07.760230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d3d2-0bac-4382-977f-2ad2aeb8e28a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:07.760230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7d3d2-0bac-4382-977f-2ad2aeb8e28a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:07.762067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 676/1352] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:07.768402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062763d-d701-40f5-a716-03f3834a8c72 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:07.768928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062763d-d701-40f5-a716-03f3834a8c72 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:07.769020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062763d-d701-40f5-a716-03f3834a8c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:07.789322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e062763d-d701-40f5-a716-03f3834a8c72 None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:07.790644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e062763d-d701-40f5-a716-03f3834a8c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:07.800512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 677/1353] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:08.222404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=13 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:09.242428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5400-b007-4dab-a84b-e17bc1b7f63f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:09.243098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5400-b007-4dab-a84b-e17bc1b7f63f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:09.243098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e5400-b007-4dab-a84b-e17bc1b7f63f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:09.244423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 677/1354] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:09.250193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce14b6-5358-493f-9039-efc5ea7eeca8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:09.250616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce14b6-5358-493f-9039-efc5ea7eeca8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:09.250756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce14b6-5358-493f-9039-efc5ea7eeca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:09.270190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bcce14b6-5358-493f-9039-efc5ea7eeca8 None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:09.271456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bcce14b6-5358-493f-9039-efc5ea7eeca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:09.281430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 678/1355] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:12.972280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83902b-eca6-4fc0-91cc-941044795459 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:12.972912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83902b-eca6-4fc0-91cc-941044795459 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:12.972912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83902b-eca6-4fc0-91cc-941044795459 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:12.974164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 678/1356] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:12.979730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe82-e96e-4935-a52c-e7bd6d000d74 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:12.980183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe82-e96e-4935-a52c-e7bd6d000d74 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:12.980270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aafe82-e96e-4935-a52c-e7bd6d000d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:13.000670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-47aafe82-e96e-4935-a52c-e7bd6d000d74 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:13.002378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-47aafe82-e96e-4935-a52c-e7bd6d000d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:13.012892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 679/1357] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:13.063616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572f98-d36e-4f2f-93dc-f1a4f3e1a17a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:13.063875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572f98-d36e-4f2f-93dc-f1a4f3e1a17a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:13.064387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572f98-d36e-4f2f-93dc-f1a4f3e1a17a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:13.066158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 679/1358] 10.222.0.74 () {54 vars in 961 bytes} [Thu Jun 16 16:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:13.083938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928ab3-4da0-45ba-a15d-2cfd81f9f8fa None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:13.084557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928ab3-4da0-45ba-a15d-2cfd81f9f8fa None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:13.085065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928ab3-4da0-45ba-a15d-2cfd81f9f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:13.131727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:13.140878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-dd928ab3-4da0-45ba-a15d-2cfd81f9f8fa None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:13.142368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dd928ab3-4da0-45ba-a15d-2cfd81f9f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:13.242562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 680/1359] 10.222.0.74 () {58 vars in 1058 bytes} [Thu Jun 16 16:33:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:13.250646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572f98-d36e-4f2f-93dc-f1a4f3e1a17a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:13.266688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572f98-d36e-4f2f-93dc-f1a4f3e1a17a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:13.272986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:13.273823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:13.274331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:13.274759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:13.283883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:13.285667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804247-0909-47e4-8d16-8bdff8995c98 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:13.296915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 680/1360] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:15.642843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5baa-428b-4e7b-a76f-d7a217e52347 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:15.643458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5baa-428b-4e7b-a76f-d7a217e52347 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:15.643458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d5baa-428b-4e7b-a76f-d7a217e52347 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:15.644981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 681/1361] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:15.650062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0129-9759-40db-8778-2fb7483b267f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:15.650558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0129-9759-40db-8778-2fb7483b267f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:15.650673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb0129-9759-40db-8778-2fb7483b267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:15.669565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-08cb0129-9759-40db-8778-2fb7483b267f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:15.670793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-08cb0129-9759-40db-8778-2fb7483b267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:15.679786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 681/1362] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:15.726938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d5baa-428b-4e7b-a76f-d7a217e52347 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:15.742314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d5baa-428b-4e7b-a76f-d7a217e52347 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:15.748219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:15.748883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:15.749165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:15.749432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:15.759331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:15.760955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a8e39-1a89-480d-872c-d78f94bfc0f0 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:15.771790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 682/1363] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:18.265306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69edc-b03c-4b2f-acf8-16fee46895b7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:18.266216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69edc-b03c-4b2f-acf8-16fee46895b7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:18.266640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f69edc-b03c-4b2f-acf8-16fee46895b7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:18.268243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 682/1364] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:18.273736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24307f-923e-4349-b0c5-74918cf473f3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:18.274547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24307f-923e-4349-b0c5-74918cf473f3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:18.274830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24307f-923e-4349-b0c5-74918cf473f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:18.293202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6a24307f-923e-4349-b0c5-74918cf473f3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:18.294409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6a24307f-923e-4349-b0c5-74918cf473f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:18.302355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 683/1365] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:18.347858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f224a-4b4c-4b52-befa-63d052a6c7c5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:18.348142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f224a-4b4c-4b52-befa-63d052a6c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:18.348442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f224a-4b4c-4b52-befa-63d052a6c7c5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:18.349823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 683/1366] 10.222.0.74 () {54 vars in 961 bytes} [Thu Jun 16 16:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:18.355411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0deed-058f-4190-aaec-c5273349c505 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:18.355702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0deed-058f-4190-aaec-c5273349c505 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:18.355987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0deed-058f-4190-aaec-c5273349c505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:18.375413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5af0deed-058f-4190-aaec-c5273349c505 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:18.376632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5af0deed-058f-4190-aaec-c5273349c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:18.387466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 684/1367] 10.222.0.74 () {58 vars in 1058 bytes} [Thu Jun 16 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:18.395755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f224a-4b4c-4b52-befa-63d052a6c7c5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:18.410788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f224a-4b4c-4b52-befa-63d052a6c7c5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:18.416571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:18.417200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:18.417495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:18.417786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:18.426243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:18.427966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277f267-357c-4fcc-a176-2c172291c129 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:18.437807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 684/1368] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:20.625391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a1e7-6b58-4921-8d99-0836e06ae224 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:20.625950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a1e7-6b58-4921-8d99-0836e06ae224 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:20.625950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a1e7-6b58-4921-8d99-0836e06ae224 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:20.627357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 685/1369] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:20.632576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a7ac-dac8-401d-97b2-32f96ea1e6bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:20.633638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a7ac-dac8-401d-97b2-32f96ea1e6bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:20.634087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a7ac-dac8-401d-97b2-32f96ea1e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:20.652303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b986a7ac-dac8-401d-97b2-32f96ea1e6bb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:20.653765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b986a7ac-dac8-401d-97b2-32f96ea1e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:20.661844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 685/1370] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:20.705921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67a1e7-6b58-4921-8d99-0836e06ae224 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:20.722074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67a1e7-6b58-4921-8d99-0836e06ae224 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:20.727787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:20.728505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:20.728790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:20.729067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:20.738037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:20.739676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a85f1d-5332-4ef3-aa99-1812c549bd69 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:20.749797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 686/1371] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:21.330308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2914-2b1c-4bd7-913a-edb66bbb7b6c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:21.330732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2914-2b1c-4bd7-913a-edb66bbb7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:21.330927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2914-2b1c-4bd7-913a-edb66bbb7b6c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:21.332378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 686/1372] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:21.337808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad634a1-8989-45e2-ad35-5497a581066e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:21.338104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad634a1-8989-45e2-ad35-5497a581066e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:21.338388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad634a1-8989-45e2-ad35-5497a581066e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:21.357610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6ad634a1-8989-45e2-ad35-5497a581066e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:21.358834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6ad634a1-8989-45e2-ad35-5497a581066e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:21.367305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 687/1373] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:21.667940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8121-a15d-4ecb-8575-ea85e596e373 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:21.667940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8121-a15d-4ecb-8575-ea85e596e373 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:21.667940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8121-a15d-4ecb-8575-ea85e596e373 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:21.669702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 687/1374] 10.222.0.74 () {54 vars in 963 bytes} [Thu Jun 16 16:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:21.674209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0982a-425c-457b-9116-d59933517519 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:21.674668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0982a-425c-457b-9116-d59933517519 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:21.674828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0982a-425c-457b-9116-d59933517519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:21.727742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bce0982a-425c-457b-9116-d59933517519 None None] MFA Rules not processed for user `1a66c447d6e943a29b2bf8a6c5309294`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:21.728956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bce0982a-425c-457b-9116-d59933517519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:21.790985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 688/1375] 10.222.0.74 () {58 vars in 1060 bytes} [Thu Jun 16 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:21.798446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8121-a15d-4ecb-8575-ea85e596e373 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:21.809515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8121-a15d-4ecb-8575-ea85e596e373 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:21.813146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:21.813812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:21.814110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:21.814571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:21.824342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:21.825786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acdb6-cf50-4cd5-9124-661b452152df service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:21.835910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 688/1376] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:22.933645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3737ab-f71f-49f4-b8f9-09d9306bbd01 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:22.933645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3737ab-f71f-49f4-b8f9-09d9306bbd01 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:22.934272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3737ab-f71f-49f4-b8f9-09d9306bbd01 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:22.935385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 689/1377] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:22.941060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84948-2530-4bb2-97ff-76a0ccc223df None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:22.941475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84948-2530-4bb2-97ff-76a0ccc223df None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:22.941632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b84948-2530-4bb2-97ff-76a0ccc223df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:22.960800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c6b84948-2530-4bb2-97ff-76a0ccc223df None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:22.962095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c6b84948-2530-4bb2-97ff-76a0ccc223df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:22.971037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 689/1378] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:23.015574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3737ab-f71f-49f4-b8f9-09d9306bbd01 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:23.030766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3737ab-f71f-49f4-b8f9-09d9306bbd01 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:23.036301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:23.036948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:23.037236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:23.037516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:23.046172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:23.047813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d858-7c9f-45ae-b946-7177548cc754 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:23.051707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:23.059390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 690/1379] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:25.204628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0c1d-0679-4732-904c-c9c32f8ad30f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:25.205690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0c1d-0679-4732-904c-c9c32f8ad30f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:25.206214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f0c1d-0679-4732-904c-c9c32f8ad30f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:25.208441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 690/1380] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:25.214116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5279d-e0cf-4897-8d14-841b5f76c8b3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:25.215247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5279d-e0cf-4897-8d14-841b5f76c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:25.215247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5279d-e0cf-4897-8d14-841b5f76c8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:25.234012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b2a5279d-e0cf-4897-8d14-841b5f76c8b3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:25.238725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b2a5279d-e0cf-4897-8d14-841b5f76c8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:25.247622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 691/1381] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:25.291992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f0c1d-0679-4732-904c-c9c32f8ad30f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:25.319655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f0c1d-0679-4732-904c-c9c32f8ad30f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:25.328065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:25.328928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:25.329391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:25.329831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:25.347218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:25.348805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2c8f8-f711-43f4-becb-62a32396640a service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:25.360611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 691/1382] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:27.547670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1e37-55a6-46e9-b73c-431c844c5345 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:27.548549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1e37-55a6-46e9-b73c-431c844c5345 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:27.549570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1e37-55a6-46e9-b73c-431c844c5345 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:27.550952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 692/1383] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:27.556572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5504-14a7-4904-9baf-3fc97ede1717 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:27.557037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5504-14a7-4904-9baf-3fc97ede1717 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:27.557135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d5504-14a7-4904-9baf-3fc97ede1717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:27.575446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-156d5504-14a7-4904-9baf-3fc97ede1717 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:27.576626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-156d5504-14a7-4904-9baf-3fc97ede1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:27.584620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 692/1384] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:27.625783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc1e37-55a6-46e9-b73c-431c844c5345 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:27.640825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc1e37-55a6-46e9-b73c-431c844c5345 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:27.646546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:27.647453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:27.647948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:27.648419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:27.657626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:27.659310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d758-1f49-404f-bcf1-bf6db2fdabb0 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:27.670778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 693/1385] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:29.779217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049f3f-db35-4cc2-93ff-edfcba782f2f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:29.779890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049f3f-db35-4cc2-93ff-edfcba782f2f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:29.779890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049f3f-db35-4cc2-93ff-edfcba782f2f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:29.781132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 693/1386] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:33:29.786786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e42a4-a1cb-45ad-ab06-ad408094d15d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:29.787240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e42a4-a1cb-45ad-ab06-ad408094d15d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:29.787421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e42a4-a1cb-45ad-ab06-ad408094d15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:29.810067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-926e42a4-a1cb-45ad-ab06-ad408094d15d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:29.811301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-926e42a4-a1cb-45ad-ab06-ad408094d15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:29.820295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 694/1387] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:29.866148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c049f3f-db35-4cc2-93ff-edfcba782f2f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:29.880804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c049f3f-db35-4cc2-93ff-edfcba782f2f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:29.887474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:29.888182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:29.888463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:29.888731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:29.897460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:29.899085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c807d-f40d-4f82-9506-bc57edc607ab service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:29.909066 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 694/1388] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:32.120906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1421ed7-eda0-4a0e-b848-78e22906cd7e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:32.121492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1421ed7-eda0-4a0e-b848-78e22906cd7e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:32.121492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1421ed7-eda0-4a0e-b848-78e22906cd7e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:32.122922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 695/1389] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:32.128974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a01a-2ddd-46ab-8c5d-f6adb0fef29e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:32.129515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a01a-2ddd-46ab-8c5d-f6adb0fef29e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:32.129515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9a01a-2ddd-46ab-8c5d-f6adb0fef29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:32.149460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bdc9a01a-2ddd-46ab-8c5d-f6adb0fef29e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:32.150734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bdc9a01a-2ddd-46ab-8c5d-f6adb0fef29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:32.159207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 695/1390] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:33:32.205933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1421ed7-eda0-4a0e-b848-78e22906cd7e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:32.220708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1421ed7-eda0-4a0e-b848-78e22906cd7e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:32.226242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:32.226917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:32.227246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:32.227522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:32.236456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:32.238089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b82a5-1048-4186-bce0-a7fbcdb9dfce service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:32.248282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 696/1391] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:34.606846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b087c-ba7e-4af6-987d-758031c26336 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:34.607486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b087c-ba7e-4af6-987d-758031c26336 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:34.607486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b087c-ba7e-4af6-987d-758031c26336 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:34.608846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 696/1392] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:34.614385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dac7d-f6af-45d3-8e56-665f7a4eb813 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:34.614826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dac7d-f6af-45d3-8e56-665f7a4eb813 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:34.615004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dac7d-f6af-45d3-8e56-665f7a4eb813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:34.637300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-839dac7d-f6af-45d3-8e56-665f7a4eb813 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:34.637985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-839dac7d-f6af-45d3-8e56-665f7a4eb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:34.647195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 697/1393] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:34.698469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b087c-ba7e-4af6-987d-758031c26336 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:34.713949 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b087c-ba7e-4af6-987d-758031c26336 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:34.719396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:34.720071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:34.720360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:34.720647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:34.730051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:34.731703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0f66a-4246-42d0-be4f-c13fdb2b42d6 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:34.741612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 697/1394] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:36.894807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612aea-2358-48b7-9999-2d0b7fc0e685 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:36.895398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612aea-2358-48b7-9999-2d0b7fc0e685 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:36.895398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e612aea-2358-48b7-9999-2d0b7fc0e685 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:36.896697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 698/1395] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:36.902191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41d84-f95e-48b5-b2a9-a3965f75612f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:36.902666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41d84-f95e-48b5-b2a9-a3965f75612f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:36.902777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca41d84-f95e-48b5-b2a9-a3965f75612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:36.922136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5ca41d84-f95e-48b5-b2a9-a3965f75612f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:36.923372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ca41d84-f95e-48b5-b2a9-a3965f75612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:36.931206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 698/1396] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:36.975316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e612aea-2358-48b7-9999-2d0b7fc0e685 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:36.991639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e612aea-2358-48b7-9999-2d0b7fc0e685 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:36.997912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:36.998559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:36.998846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:36.999162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:37.008166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:37.009773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfbd1c-2f85-45dd-a5da-b0c482da23a0 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:37.020764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 699/1397] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:38.222295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616408ea-4d01-4741-ace2-fcd3f4d62a73 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:38.222921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616408ea-4d01-4741-ace2-fcd3f4d62a73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:38.222921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616408ea-4d01-4741-ace2-fcd3f4d62a73 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:38.224300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 699/1398] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:38.230283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c08c-40f9-4caa-8944-bf6fa50c32c4 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:38.230727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c08c-40f9-4caa-8944-bf6fa50c32c4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:38.230874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c08c-40f9-4caa-8944-bf6fa50c32c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:38.250376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8114c08c-40f9-4caa-8944-bf6fa50c32c4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:38.251635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8114c08c-40f9-4caa-8944-bf6fa50c32c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:38.261126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 700/1399] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:38.317664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da7a4-ee1a-41bc-8036-a0c81669843a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:38.317952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da7a4-ee1a-41bc-8036-a0c81669843a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:38.318190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73da7a4-ee1a-41bc-8036-a0c81669843a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:38.319798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 700/1400] 10.222.0.74 () {54 vars in 962 bytes} [Thu Jun 16 16:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:33:38.325090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a33d9-0ed9-4280-8eff-fead6e035678 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:38.325404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a33d9-0ed9-4280-8eff-fead6e035678 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:38.325891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a33d9-0ed9-4280-8eff-fead6e035678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:38.378134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-517a33d9-0ed9-4280-8eff-fead6e035678 None None] MFA Rules not processed for user `14ce6a5053f24cff8997908c6d7948a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:38.379362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-517a33d9-0ed9-4280-8eff-fead6e035678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:38.440817 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 701/1401] 10.222.0.74 () {58 vars in 1059 bytes} [Thu Jun 16 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:38.448728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73da7a4-ee1a-41bc-8036-a0c81669843a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:38.454140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=55 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:38.461101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73da7a4-ee1a-41bc-8036-a0c81669843a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:38.465720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:38.466367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:38.466649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:38.466923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:38.476835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:38.478243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60025cb4-d5f0-488a-a23d-83685cbd8c1e service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:38.488931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 701/1402] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:39.230673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fda8-eb89-42a6-b25c-ae91fa13ee14 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.231644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fda8-eb89-42a6-b25c-ae91fa13ee14 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.231644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fda8-eb89-42a6-b25c-ae91fa13ee14 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.233954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 702/1403] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:33:39.240645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.241174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.241416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.265527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:39.266734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:39.276548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 702/1404] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:33:39.282018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db90d34-1442-49ee-bb72-e02252d17280 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.282311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db90d34-1442-49ee-bb72-e02252d17280 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.282605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db90d34-1442-49ee-bb72-e02252d17280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.328442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:39.346683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413a223-a182-4a94-9cf3-8bc14c7743fd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:39.349721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6db90d34-1442-49ee-bb72-e02252d17280 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:39.351298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6db90d34-1442-49ee-bb72-e02252d17280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:39.352817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:39.353530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.354336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.354336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.366334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:39.369521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53764d45-f122-467f-8e1e-750f0a340663 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:39.385884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 703/1405] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:33:39.402511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:39.433489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 703/1406] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:39.488975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d076121-f355-4770-9e87-5c374ab98491 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.489453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d076121-f355-4770-9e87-5c374ab98491 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.489940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d076121-f355-4770-9e87-5c374ab98491 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.492724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 704/1407] 10.222.0.74 () {54 vars in 942 bytes} [Thu Jun 16 16:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:33:39.500912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ea7f-ad4e-4ccd-bc38-192796fa3467 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.501201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ea7f-ad4e-4ccd-bc38-192796fa3467 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.501476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ea7f-ad4e-4ccd-bc38-192796fa3467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.577983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8026ea7f-ad4e-4ccd-bc38-192796fa3467 None None] MFA Rules not processed for user `3fc2a725e2f14b90be5575a030f47dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:33:39.579292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8026ea7f-ad4e-4ccd-bc38-192796fa3467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:39.676945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 704/1408] 10.222.0.74 () {58 vars in 1039 bytes} [Thu Jun 16 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:33:39.685925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d076121-f355-4770-9e87-5c374ab98491 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:39.700505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d076121-f355-4770-9e87-5c374ab98491 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:39.707325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:39.708047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:39.708330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:39.708606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:39.717867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:39.719495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3dd90-71a5-4125-9da2-7725d937c9f4 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:39.727984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 705/1409] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:33:49.670079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=53 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:49.724563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=20 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:01.986899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170bbea-774c-43e9-a1f9-20214f1a965b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:01.987525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170bbea-774c-43e9-a1f9-20214f1a965b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:01.987525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170bbea-774c-43e9-a1f9-20214f1a965b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:01.988872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 705/1410] 10.222.0.74 () {54 vars in 918 bytes} [Thu Jun 16 16:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:01.994805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f9a-c57c-4503-8cf8-58f207b629bc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:01.995603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f9a-c57c-4503-8cf8-58f207b629bc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:01.995779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84f9a-c57c-4503-8cf8-58f207b629bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:02.025313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6e84f9a-c57c-4503-8cf8-58f207b629bc None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:02.026509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6e84f9a-c57c-4503-8cf8-58f207b629bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:02.036198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 706/1411] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Jun 16 16:34:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:02.051583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6106-54d8-40a6-98c3-3621ab8293bf None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:02.051863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6106-54d8-40a6-98c3-3621ab8293bf None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:02.052213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa6106-54d8-40a6-98c3-3621ab8293bf None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:02.054153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 706/1412] 10.222.0.74 () {54 vars in 918 bytes} [Thu Jun 16 16:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:02.059100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553c32-90b0-46a9-8657-c326c162768f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:02.059434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553c32-90b0-46a9-8657-c326c162768f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:02.059711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44553c32-90b0-46a9-8657-c326c162768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:02.079475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-44553c32-90b0-46a9-8657-c326c162768f None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:02.080682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44553c32-90b0-46a9-8657-c326c162768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:02.091544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 707/1413] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Jun 16 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:04.695374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53fdad-edce-4290-a6e4-d1c4c48757a3 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:04.696304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53fdad-edce-4290-a6e4-d1c4c48757a3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:04.696598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53fdad-edce-4290-a6e4-d1c4c48757a3 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:04.698037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:04.703317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5f12-e5d2-4bd6-a816-325bc2b3cece None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:04.703734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5f12-e5d2-4bd6-a816-325bc2b3cece None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:04.703904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d5f12-e5d2-4bd6-a816-325bc2b3cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:04.722397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-080d5f12-e5d2-4bd6-a816-325bc2b3cece None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:04.723668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-080d5f12-e5d2-4bd6-a816-325bc2b3cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:04.732323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:04.776519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53fdad-edce-4290-a6e4-d1c4c48757a3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:04.792735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53fdad-edce-4290-a6e4-d1c4c48757a3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:04.798376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:04.799089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:04.799325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:04.799598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:04.808834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:04.810445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c184-b3de-4433-9362-535709f7b7b4 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:04.820258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 708/1416] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:05.778963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fe96-1df2-4440-964c-3a2f8a1357f4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:05.779568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fe96-1df2-4440-964c-3a2f8a1357f4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:05.779568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fe96-1df2-4440-964c-3a2f8a1357f4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:05.780965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:05.786869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1802f9c-7745-4551-a3f8-da95c454f7bb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:05.787356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1802f9c-7745-4551-a3f8-da95c454f7bb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:05.787453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1802f9c-7745-4551-a3f8-da95c454f7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:05.810875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e1802f9c-7745-4551-a3f8-da95c454f7bb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:05.812051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e1802f9c-7745-4551-a3f8-da95c454f7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:05.821913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:05.978389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9fe96-1df2-4440-964c-3a2f8a1357f4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:05.994563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9fe96-1df2-4440-964c-3a2f8a1357f4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:06.000767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:06.001412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:06.001696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:06.001978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:06.011575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:06.013175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35319df1-8584-4133-a9d5-f6e6452c6b09 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:06.028151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 710/1419] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:06.965923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8452a6-806a-4bc7-bd12-0b2b1deea053 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:06.966803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8452a6-806a-4bc7-bd12-0b2b1deea053 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:06.966803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8452a6-806a-4bc7-bd12-0b2b1deea053 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:06.969065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:06.977301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f05e7-1b0f-44e1-b1ee-743b980e784e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:06.977730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f05e7-1b0f-44e1-b1ee-743b980e784e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:06.977865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f05e7-1b0f-44e1-b1ee-743b980e784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:06.997316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0b3f05e7-1b0f-44e1-b1ee-743b980e784e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:06.998542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0b3f05e7-1b0f-44e1-b1ee-743b980e784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:07.008359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:07.161117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8452a6-806a-4bc7-bd12-0b2b1deea053 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:07.182985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8452a6-806a-4bc7-bd12-0b2b1deea053 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:07.189397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:07.190034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:07.190365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:07.190670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:07.200871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:07.202525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826e950-5f92-4ee9-bf3b-a2664c829497 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:07.212652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 711/1422] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:08.116896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826eb85-bc68-4c7b-8fb2-8fc8f5827e3f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:08.117671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826eb85-bc68-4c7b-8fb2-8fc8f5827e3f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:08.117822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826eb85-bc68-4c7b-8fb2-8fc8f5827e3f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:08.120244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:08.125617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4952-3007-4a46-a4dd-8629c5069b5d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:08.126059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4952-3007-4a46-a4dd-8629c5069b5d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:08.126195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e4952-3007-4a46-a4dd-8629c5069b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:08.146823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5e1e4952-3007-4a46-a4dd-8629c5069b5d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:08.148085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5e1e4952-3007-4a46-a4dd-8629c5069b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:08.158728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:08.319250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826eb85-bc68-4c7b-8fb2-8fc8f5827e3f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:08.334996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826eb85-bc68-4c7b-8fb2-8fc8f5827e3f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:08.340361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:08.340993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:08.341272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:08.341548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:08.349659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:08.351310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3d533-7941-4415-a115-4ede0038e2f8 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:08.360707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 713/1425] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:09.269350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cea-e2de-4e07-840e-740a4b40a427 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:09.270253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cea-e2de-4e07-840e-740a4b40a427 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:09.270253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136cea-e2de-4e07-840e-740a4b40a427 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:09.272210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:09.277594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f9a0-82a5-4e18-a047-64d1931b1f6d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:09.278070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f9a0-82a5-4e18-a047-64d1931b1f6d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:09.278212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8f9a0-82a5-4e18-a047-64d1931b1f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:09.303871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-efc8f9a0-82a5-4e18-a047-64d1931b1f6d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:09.305246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-efc8f9a0-82a5-4e18-a047-64d1931b1f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:09.313713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:09.465570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136cea-e2de-4e07-840e-740a4b40a427 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:09.483691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136cea-e2de-4e07-840e-740a4b40a427 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:09.490866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:09.491547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:09.491924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:09.492324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:09.501239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:09.502855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e0120-e510-4340-9940-82c7ef7cea95 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:09.513291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 714/1428] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:10.407925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46c8b-b046-43e9-a94f-645b2f697eea None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:10.408819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46c8b-b046-43e9-a94f-645b2f697eea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:10.408819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46c8b-b046-43e9-a94f-645b2f697eea None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:10.410948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:10.417423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289e6d-3d15-4ee7-adb4-b1aaf1b0bb1b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:10.417860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289e6d-3d15-4ee7-adb4-b1aaf1b0bb1b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:10.417970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289e6d-3d15-4ee7-adb4-b1aaf1b0bb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:10.437490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-07289e6d-3d15-4ee7-adb4-b1aaf1b0bb1b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:10.438705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-07289e6d-3d15-4ee7-adb4-b1aaf1b0bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:10.449649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:10.606172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c46c8b-b046-43e9-a94f-645b2f697eea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:10.624229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c46c8b-b046-43e9-a94f-645b2f697eea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:10.630821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:10.631509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:10.631795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:10.632078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:10.641176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:10.642770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3b17-1e56-41d0-985f-e0b38387e6a8 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:10.653005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 716/1431] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:11.505165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d85a-a85a-42b8-a832-2c24d019b939 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:11.506473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d85a-a85a-42b8-a832-2c24d019b939 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:11.506473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89d85a-a85a-42b8-a832-2c24d019b939 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:11.509024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:11.514288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3767c-b21b-4811-8e90-574f746fb5ad None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:11.514718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3767c-b21b-4811-8e90-574f746fb5ad None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:11.514833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3767c-b21b-4811-8e90-574f746fb5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:11.534461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cec3767c-b21b-4811-8e90-574f746fb5ad None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:11.535709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cec3767c-b21b-4811-8e90-574f746fb5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:11.547100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:11.698461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89d85a-a85a-42b8-a832-2c24d019b939 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:11.718931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89d85a-a85a-42b8-a832-2c24d019b939 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:11.724583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:11.725230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:11.725519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:11.726275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:11.734847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:11.736503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b4023-4287-411b-a858-b305bca3193b service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:11.746456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 717/1434] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:12.594126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195abc-3818-47ce-858b-ae5cddcea76d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:12.594720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195abc-3818-47ce-858b-ae5cddcea76d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:12.594720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9195abc-3818-47ce-858b-ae5cddcea76d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:12.596070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:12.601310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26efcc-f2b0-41c8-a034-04d9d1a325fb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:12.601769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26efcc-f2b0-41c8-a034-04d9d1a325fb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:12.601981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26efcc-f2b0-41c8-a034-04d9d1a325fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:12.621826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1b26efcc-f2b0-41c8-a034-04d9d1a325fb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:12.623521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1b26efcc-f2b0-41c8-a034-04d9d1a325fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:12.632722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:12.782098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195abc-3818-47ce-858b-ae5cddcea76d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:12.798013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9195abc-3818-47ce-858b-ae5cddcea76d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:12.803884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:12.804493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:12.804777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:12.805054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:12.813957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:12.815583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911910-1b39-4d5f-a093-d044c697c4fa service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:12.825638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 719/1437] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:13.703582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30112-301f-4e79-9a38-b9b590107c4c None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:13.704241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30112-301f-4e79-9a38-b9b590107c4c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:13.704241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30112-301f-4e79-9a38-b9b590107c4c None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:13.705626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:13.711501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fdf09-afba-405e-ad4c-0e91f672ec1b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:13.712065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fdf09-afba-405e-ad4c-0e91f672ec1b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:13.712065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fdf09-afba-405e-ad4c-0e91f672ec1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:13.738544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-424fdf09-afba-405e-ad4c-0e91f672ec1b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:13.740371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-424fdf09-afba-405e-ad4c-0e91f672ec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:13.753424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:13.912853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30112-301f-4e79-9a38-b9b590107c4c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:13.929430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30112-301f-4e79-9a38-b9b590107c4c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:13.935331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:13.935981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:13.937005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:13.937296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:13.946636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:13.948252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b398d3-5b68-4f8e-9de2-38662aa448a0 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:13.958306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 720/1440] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:16.015617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075e5a-19e1-452a-a138-bccb693e24d5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:16.016246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075e5a-19e1-452a-a138-bccb693e24d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:16.016246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0075e5a-19e1-452a-a138-bccb693e24d5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:16.017570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:16.023301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9b54b-ad12-4ad2-ba74-6879e5ae440f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:16.023745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9b54b-ad12-4ad2-ba74-6879e5ae440f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:16.023865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9b54b-ad12-4ad2-ba74-6879e5ae440f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:16.042436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-14e9b54b-ad12-4ad2-ba74-6879e5ae440f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:16.043760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-14e9b54b-ad12-4ad2-ba74-6879e5ae440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:16.052851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:16.094141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0075e5a-19e1-452a-a138-bccb693e24d5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:16.107724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0075e5a-19e1-452a-a138-bccb693e24d5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:16.111859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:16.112512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:16.112788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:16.113057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:16.123038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:16.124503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7612-c3fd-4ff0-817e-87e36ee44e33 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:16.134842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 722/1443] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:18.486479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d9162-1d51-4a40-bf2b-66a481eb340d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:18.487129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d9162-1d51-4a40-bf2b-66a481eb340d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:18.487129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d9162-1d51-4a40-bf2b-66a481eb340d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:18.488452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:18.493989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15582-e558-4303-bfe2-38a6f6f703ce None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:18.494442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15582-e558-4303-bfe2-38a6f6f703ce None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:18.494547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15582-e558-4303-bfe2-38a6f6f703ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:18.513698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2cc15582-e558-4303-bfe2-38a6f6f703ce None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:18.514900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2cc15582-e558-4303-bfe2-38a6f6f703ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:18.524015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:18.561350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d9162-1d51-4a40-bf2b-66a481eb340d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:18.574050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d9162-1d51-4a40-bf2b-66a481eb340d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:18.578656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:18.579484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:18.579787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:18.580061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:18.589778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:18.591263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412c855-891f-4591-a8eb-3e31fd09b583 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:18.601511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 723/1446] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:21.026825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792e374-29e4-47a8-a45a-73e6bf1d6878 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:21.027463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792e374-29e4-47a8-a45a-73e6bf1d6878 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:21.027463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792e374-29e4-47a8-a45a-73e6bf1d6878 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:21.028834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:21.034276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626e21-f050-4fd1-8063-cdf60ab1710e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:21.034714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626e21-f050-4fd1-8063-cdf60ab1710e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:21.034836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626e21-f050-4fd1-8063-cdf60ab1710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:21.053839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-33626e21-f050-4fd1-8063-cdf60ab1710e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:21.055075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-33626e21-f050-4fd1-8063-cdf60ab1710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:21.063340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:21.102062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792e374-29e4-47a8-a45a-73e6bf1d6878 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:21.114425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792e374-29e4-47a8-a45a-73e6bf1d6878 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:21.119269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:21.120124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:21.120443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:21.120760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:21.132138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:21.133586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b30423-9bbb-44e0-959d-ef15ff3169fa service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:21.143521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 725/1449] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:23.385298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076056db-c5cd-47bf-9d42-1eaf3761388b None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:23.385896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076056db-c5cd-47bf-9d42-1eaf3761388b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:23.385896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076056db-c5cd-47bf-9d42-1eaf3761388b None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:23.387229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:23.392663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13b11-6809-4b71-b2d8-2fe74fbba3d0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:23.393110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13b11-6809-4b71-b2d8-2fe74fbba3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:23.393239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f13b11-6809-4b71-b2d8-2fe74fbba3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:23.413250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-40f13b11-6809-4b71-b2d8-2fe74fbba3d0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:23.414554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-40f13b11-6809-4b71-b2d8-2fe74fbba3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:23.423915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:23.463396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076056db-c5cd-47bf-9d42-1eaf3761388b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:23.474639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076056db-c5cd-47bf-9d42-1eaf3761388b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:23.478376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:23.479011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:23.479323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:23.479618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:23.488414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:23.489846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38edc2-aaf8-47d4-a473-01df11a4398c service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:23.499568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 726/1452] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:25.722781 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5599f7-3384-4c4a-8205-dcc0d60be431 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:25.723435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5599f7-3384-4c4a-8205-dcc0d60be431 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:25.723435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5599f7-3384-4c4a-8205-dcc0d60be431 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:25.724760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:25.730387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae6dc-5aad-48d8-a58c-6e33f41b8344 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:25.730869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae6dc-5aad-48d8-a58c-6e33f41b8344 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:25.730978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae6dc-5aad-48d8-a58c-6e33f41b8344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:25.750225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6e8ae6dc-5aad-48d8-a58c-6e33f41b8344 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:25.751488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6e8ae6dc-5aad-48d8-a58c-6e33f41b8344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:25.761142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:25.799598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5599f7-3384-4c4a-8205-dcc0d60be431 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:25.812805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5599f7-3384-4c4a-8205-dcc0d60be431 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:25.817337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:25.817989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:25.818273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:25.818547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:25.829299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:25.830752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b855ea-8956-4f0d-b59d-d1817319de9c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:25.841554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 728/1455] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:28.000876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb70e-b05a-4f40-a6bc-fa0fb7ec899a None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:28.001489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb70e-b05a-4f40-a6bc-fa0fb7ec899a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:28.001489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb70e-b05a-4f40-a6bc-fa0fb7ec899a None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:28.002797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:28.008736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4394e-0703-4b0f-9d31-aba096009d84 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:28.009184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4394e-0703-4b0f-9d31-aba096009d84 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:28.009300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4394e-0703-4b0f-9d31-aba096009d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:28.028935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7df4394e-0703-4b0f-9d31-aba096009d84 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:28.030165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7df4394e-0703-4b0f-9d31-aba096009d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:28.039591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:28.082851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bb70e-b05a-4f40-a6bc-fa0fb7ec899a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:28.094748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bb70e-b05a-4f40-a6bc-fa0fb7ec899a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:28.098589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:28.099264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:28.099549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:28.099830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:28.110322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:28.111814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf29b1-6363-4fe0-8849-9d1c4f34c2a8 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:28.123461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 729/1458] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:30.399138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa35843-cce6-43b5-b5b1-c40fa5d89dec None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:30.399718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa35843-cce6-43b5-b5b1-c40fa5d89dec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:30.399718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa35843-cce6-43b5-b5b1-c40fa5d89dec None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:30.400929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:30.406892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc6e4-0f55-433c-b7e8-c34e9e8bb882 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:30.407349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc6e4-0f55-433c-b7e8-c34e9e8bb882 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:30.407593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc6e4-0f55-433c-b7e8-c34e9e8bb882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:30.414516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=70 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:30.430215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c79bc6e4-0f55-433c-b7e8-c34e9e8bb882 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:30.431501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c79bc6e4-0f55-433c-b7e8-c34e9e8bb882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:30.440349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:30.478071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa35843-cce6-43b5-b5b1-c40fa5d89dec None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:30.484099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=73 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:30.492723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa35843-cce6-43b5-b5b1-c40fa5d89dec None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:30.496567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:30.497225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:30.497510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:30.497786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:30.508587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:30.510044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4b8a1-24f3-4b60-8df6-1caa19121af2 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:30.520159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 731/1461] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:32.700350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e901e6-f4a8-4a40-a574-bc4abe467644 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:32.700935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e901e6-f4a8-4a40-a574-bc4abe467644 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:32.700935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e901e6-f4a8-4a40-a574-bc4abe467644 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:32.702306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:32.708384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69abd-5187-4c1f-99d7-cac294522f1a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:32.708855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69abd-5187-4c1f-99d7-cac294522f1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:32.708984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69abd-5187-4c1f-99d7-cac294522f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:32.729058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8bf69abd-5187-4c1f-99d7-cac294522f1a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:32.730284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8bf69abd-5187-4c1f-99d7-cac294522f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:32.741638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:34:32.791307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e901e6-f4a8-4a40-a574-bc4abe467644 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:32.802695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e901e6-f4a8-4a40-a574-bc4abe467644 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:32.806758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:32.807441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:32.807721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:32.807988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:32.818816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:32.820320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512d6cd-5c0c-46f9-98d1-72f197be2166 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:32.830985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 732/1464] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:35.005963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af429bd-6418-4f13-bfe3-19db8eec42c4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:35.006575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af429bd-6418-4f13-bfe3-19db8eec42c4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:35.006575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af429bd-6418-4f13-bfe3-19db8eec42c4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:35.007951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:35.013648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d653a0-089a-4497-a86c-e12824445946 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:35.014138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d653a0-089a-4497-a86c-e12824445946 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:35.014241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d653a0-089a-4497-a86c-e12824445946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:35.033736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-71d653a0-089a-4497-a86c-e12824445946 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:35.034974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-71d653a0-089a-4497-a86c-e12824445946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:35.043861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:35.081494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af429bd-6418-4f13-bfe3-19db8eec42c4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:35.093580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af429bd-6418-4f13-bfe3-19db8eec42c4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:35.097283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:35.097926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:35.098206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:35.098479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:35.110486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:35.111991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8e77b-5a6c-43ff-8be5-84d4abecaa97 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:35.123778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 734/1467] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:37.344927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf01bc3-4e4d-4e9c-b843-e3321e14e1e5 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:37.345495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf01bc3-4e4d-4e9c-b843-e3321e14e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:37.345495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf01bc3-4e4d-4e9c-b843-e3321e14e1e5 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:37.346867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:37.353571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3d07-3947-4418-825e-b8f6b73fcdd2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:37.354037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3d07-3947-4418-825e-b8f6b73fcdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:37.354135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c3d07-3947-4418-825e-b8f6b73fcdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:37.378907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-186c3d07-3947-4418-825e-b8f6b73fcdd2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:37.380941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-186c3d07-3947-4418-825e-b8f6b73fcdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:37.389507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:37.430025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf01bc3-4e4d-4e9c-b843-e3321e14e1e5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:37.441804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf01bc3-4e4d-4e9c-b843-e3321e14e1e5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:37.446192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:37.446859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:37.447182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:37.447462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:37.457502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:37.459825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11444405-99b9-41ed-81dd-a8e22fd01f1a service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:37.469948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 735/1470] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:47.371717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=17 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:47.465376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=17 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:57.844912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbe22-73c4-4904-8362-a4ced2502021 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:57.845518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbe22-73c4-4904-8362-a4ced2502021 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:57.845518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfbe22-73c4-4904-8362-a4ced2502021 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:57.846838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 736/1471] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:57.853603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e5b11-d0b5-4c54-afb0-3cbaf262c78e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:57.854116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e5b11-d0b5-4c54-afb0-3cbaf262c78e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:57.854116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e5b11-d0b5-4c54-afb0-3cbaf262c78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:57.875944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a07e5b11-d0b5-4c54-afb0-3cbaf262c78e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:57.877201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a07e5b11-d0b5-4c54-afb0-3cbaf262c78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:57.891131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 736/1472] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:34:57.942499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3bde4-da5e-403e-afd3-c69b16983468 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:57.942880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3bde4-da5e-403e-afd3-c69b16983468 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:57.943193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3bde4-da5e-403e-afd3-c69b16983468 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:57.944561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 737/1473] 10.222.0.74 () {54 vars in 962 bytes} [Thu Jun 16 16:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:34:57.949871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047c18-86d3-4f04-ad3f-8559b2a78844 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:57.950240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047c18-86d3-4f04-ad3f-8559b2a78844 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:57.950534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047c18-86d3-4f04-ad3f-8559b2a78844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:57.969573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-96047c18-86d3-4f04-ad3f-8559b2a78844 None None] MFA Rules not processed for user `14ce6a5053f24cff8997908c6d7948a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:57.970768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-96047c18-86d3-4f04-ad3f-8559b2a78844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:57.977780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 737/1474] 10.222.0.74 () {58 vars in 1059 bytes} [Thu Jun 16 16:34:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:57.985283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3bde4-da5e-403e-afd3-c69b16983468 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:57.998003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3bde4-da5e-403e-afd3-c69b16983468 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:58.002080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04527523-405f-4a81-af39-c9949be18257 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:58.002924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527523-405f-4a81-af39-c9949be18257 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:58.003235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527523-405f-4a81-af39-c9949be18257 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:58.003511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527523-405f-4a81-af39-c9949be18257 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:58.014627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527523-405f-4a81-af39-c9949be18257 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:58.016096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527523-405f-4a81-af39-c9949be18257 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:58.026914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 738/1475] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:00.243882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a322-91a8-4ddd-abcd-5d1d8a1fc901 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:00.244458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a322-91a8-4ddd-abcd-5d1d8a1fc901 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:00.244458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a322-91a8-4ddd-abcd-5d1d8a1fc901 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:00.245828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 738/1476] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:00.251361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119c58-35a6-43ed-a479-b34d60e30ad4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:00.251757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119c58-35a6-43ed-a479-b34d60e30ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:00.251919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119c58-35a6-43ed-a479-b34d60e30ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:00.271844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5c119c58-35a6-43ed-a479-b34d60e30ad4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:00.272999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5c119c58-35a6-43ed-a479-b34d60e30ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:00.281464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 739/1477] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:00.306354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889a322-91a8-4ddd-abcd-5d1d8a1fc901 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:00.320734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889a322-91a8-4ddd-abcd-5d1d8a1fc901 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:00.325829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:00.326527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:00.326810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:00.327151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:00.327958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:00.329857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:35:00.329857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:35:00.329857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:35:00.331084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84422fe2-81e5-4e5c-9e0f-52c1820638a0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:00.352476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 739/1478] 10.222.0.74 () {56 vars in 1108 bytes} [Thu Jun 16 16:35:00 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:35:02.567164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5f0d8-c5e5-435e-8b50-a22c538e2481 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:02.568051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5f0d8-c5e5-435e-8b50-a22c538e2481 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:02.568051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5f0d8-c5e5-435e-8b50-a22c538e2481 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:02.569072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 740/1479] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:02.574737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d22eb-307e-476c-b3c9-96eaad89dcb5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:02.575287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d22eb-307e-476c-b3c9-96eaad89dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:02.575608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d22eb-307e-476c-b3c9-96eaad89dcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:02.597489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d08d22eb-307e-476c-b3c9-96eaad89dcb5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:02.599651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d08d22eb-307e-476c-b3c9-96eaad89dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:02.609146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 740/1480] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:02.655743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5f0d8-c5e5-435e-8b50-a22c538e2481 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:02.671763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5f0d8-c5e5-435e-8b50-a22c538e2481 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:02.679888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:02.680537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:02.680826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:02.681100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:02.690135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:02.693108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607bf78-4af3-495f-b30c-0c96d09bb4c8 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:02.708320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 741/1481] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:05.148262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851452-cf18-4111-a3cb-6d03fd4735fd None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:05.148876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851452-cf18-4111-a3cb-6d03fd4735fd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:05.148876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a851452-cf18-4111-a3cb-6d03fd4735fd None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:05.150212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 741/1482] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:05.155544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81e9a-4741-4588-9823-b1133f3bf25f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:05.155996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81e9a-4741-4588-9823-b1133f3bf25f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:05.156143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81e9a-4741-4588-9823-b1133f3bf25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:05.175195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b0b81e9a-4741-4588-9823-b1133f3bf25f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:05.176544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b0b81e9a-4741-4588-9823-b1133f3bf25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:05.185503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 742/1483] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:05.222649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851452-cf18-4111-a3cb-6d03fd4735fd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:05.242314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a851452-cf18-4111-a3cb-6d03fd4735fd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:05.247036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:05.247705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:05.247984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:05.248252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:05.258598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:05.260096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ee0a3-5516-4bd5-a48d-f57a9e4e7c5f service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:05.269690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 742/1484] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:07.530341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810d958-5517-4c40-981b-4f3e1a30252b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:07.530941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810d958-5517-4c40-981b-4f3e1a30252b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:07.530941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810d958-5517-4c40-981b-4f3e1a30252b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:07.532299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 743/1485] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:07.538080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0053-48bd-459a-af42-8e81597c98f2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:07.538585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0053-48bd-459a-af42-8e81597c98f2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:07.538585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c0053-48bd-459a-af42-8e81597c98f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:07.558835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-263c0053-48bd-459a-af42-8e81597c98f2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:07.560108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-263c0053-48bd-459a-af42-8e81597c98f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:07.569253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 743/1486] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:07.608769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810d958-5517-4c40-981b-4f3e1a30252b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:07.621402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810d958-5517-4c40-981b-4f3e1a30252b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:07.625570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:07.626228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:07.626513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:07.626790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:07.637490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:07.638943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27985f2c-c557-42a0-bd96-de503472fc13 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:07.649122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 744/1487] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:10.032383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8022f8e-2667-408a-ae3e-3727fbd9fd75 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:10.032927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8022f8e-2667-408a-ae3e-3727fbd9fd75 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:10.032927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8022f8e-2667-408a-ae3e-3727fbd9fd75 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:10.034346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 744/1488] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:10.040006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7008c-89b1-4ab3-bf00-4511eb5cd73e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:10.040451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7008c-89b1-4ab3-bf00-4511eb5cd73e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:10.040530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7008c-89b1-4ab3-bf00-4511eb5cd73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:10.060030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-73f7008c-89b1-4ab3-bf00-4511eb5cd73e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:10.061260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-73f7008c-89b1-4ab3-bf00-4511eb5cd73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:10.070633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 745/1489] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:10.112190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8022f8e-2667-408a-ae3e-3727fbd9fd75 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:10.125481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8022f8e-2667-408a-ae3e-3727fbd9fd75 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:10.130329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:10.131005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:10.131322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:10.131582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:10.141592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:10.143008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd301b-ad0b-4e5c-9df9-e35df1cef74a service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:10.153318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 745/1490] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:12.504819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225c5a-6fe2-4d6e-a3d5-a65059cdd2b4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:12.505461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225c5a-6fe2-4d6e-a3d5-a65059cdd2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:12.505461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9225c5a-6fe2-4d6e-a3d5-a65059cdd2b4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:12.506748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 746/1491] 10.222.0.74 () {54 vars in 926 bytes} [Thu Jun 16 16:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:35:12.512715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807182d5-f8d3-4f00-9ab8-43359df8ce3d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:12.513299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807182d5-f8d3-4f00-9ab8-43359df8ce3d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:12.513299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807182d5-f8d3-4f00-9ab8-43359df8ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:12.536313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-807182d5-f8d3-4f00-9ab8-43359df8ce3d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:35:12.537723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-807182d5-f8d3-4f00-9ab8-43359df8ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:35:12.547169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 746/1492] 10.222.0.74 () {58 vars in 1023 bytes} [Thu Jun 16 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:35:12.595845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225c5a-6fe2-4d6e-a3d5-a65059cdd2b4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:35:12.611499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9225c5a-6fe2-4d6e-a3d5-a65059cdd2b4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:35:12.617351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:35:12.618196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:35:12.618485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:35:12.618833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:35:12.632789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:35:12.634256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4c5cf-ac4a-4bc9-b285-77ff3b22e873 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:35:12.644807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 747/1493] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:35:22.528256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=33 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:35:22.639932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=33 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:41.180263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.181100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.181100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.203353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:41.205054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:41.214510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 747/1494] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:36:41.230847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.231726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.231921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.259303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:41.260612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:41.269801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 748/1495] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:36:41.274889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.285294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daae69-3b51-4356-b0e0-a14b6fe96862 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.290896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.291629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.292021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.292358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.293123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:36:41.293464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.294609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.294609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:36:41.294609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:36:41.295582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.311785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 748/1496] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:36:41.321374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.336959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08b363-f808-4c5b-bc1f-61ff1b114665 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.345655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.346444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.346750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.347023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.347812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.350068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.350068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:36:41.350068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:36:41.351444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.395654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 749/1497] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:36:41.402651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.415960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fc8ff-53ea-49b8-9ecc-42e79f6d256a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.487592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.488269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.488610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.488931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.489622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.490642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.490642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:36:41.490642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:36:41.491861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.514477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 749/1498] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:36:41.519951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.530288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70ac87-4b01-4b5e-a4ed-f1a5ebd38e81 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.535748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.536430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.536696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.536992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.537728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.538765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.538765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:36:41.538765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:36:41.539689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.556112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 750/1499] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:36:41.560476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.572576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02e917-da7c-4c74-90da-8ca042e75b43 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.577603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.578360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.578649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.578916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.579680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.580698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.580698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:36:41.580698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:36:41.581597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8027523-011a-42b9-b903-a02c00c14e88 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.595285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 750/1500] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:36:41.600335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.614635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61151e5e-949f-4922-ad89-f3c0600dc1f7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.620374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.621261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.621539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.621813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] PATH_INFO: `/v3/projects/b45e991e94644fa594765946422204b6/users/c6015e204332414b946abf9e58754342/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.638862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45e991e94644fa594765946422204b6, user_id=c6015e204332414b946abf9e58754342, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.639950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:41.639950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:36:41.639950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:36:41.641888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7550ff-66c5-49d0-9df5-729a90fd010d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.660517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 751/1501] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:36:41 2022] PUT /identity/v3/projects/b45e991e94644fa594765946422204b6/users/c6015e204332414b946abf9e58754342/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:36:41.675962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200bb8-6864-46f7-9cd6-6304be647d53 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.676253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200bb8-6864-46f7-9cd6-6304be647d53 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.676532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200bb8-6864-46f7-9cd6-6304be647d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.705635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ad200bb8-6864-46f7-9cd6-6304be647d53 None None] MFA Rules not processed for user `c6015e204332414b946abf9e58754342`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:41.706904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ad200bb8-6864-46f7-9cd6-6304be647d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:41.776235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 751/1502] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:36:41.789531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3f2b-2ad0-48e1-aba5-94f954517302 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.789845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3f2b-2ad0-48e1-aba5-94f954517302 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.790088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3f2b-2ad0-48e1-aba5-94f954517302 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.791513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 752/1503] 10.222.0.74 () {54 vars in 963 bytes} [Thu Jun 16 16:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:36:41.796808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152f2d-33e5-4721-9d92-cecf83157ddb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.797184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152f2d-33e5-4721-9d92-cecf83157ddb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.797726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152f2d-33e5-4721-9d92-cecf83157ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.817907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a4152f2d-33e5-4721-9d92-cecf83157ddb None None] MFA Rules not processed for user `1a66c447d6e943a29b2bf8a6c5309294`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:41.819187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a4152f2d-33e5-4721-9d92-cecf83157ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:41.867578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:41.877880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 752/1504] 10.222.0.74 () {58 vars in 1060 bytes} [Thu Jun 16 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:36:41.886466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3f2b-2ad0-48e1-aba5-94f954517302 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:41.899023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3f2b-2ad0-48e1-aba5-94f954517302 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:41.903096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:41.903734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.904025 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.904305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.913543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:41.915161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:41.924732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 753/1505] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:36:41.970489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdcec5-ce9c-42d0-bc52-4a390b071f75 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:41.970823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdcec5-ce9c-42d0-bc52-4a390b071f75 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:41.971234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdcec5-ce9c-42d0-bc52-4a390b071f75 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:41.972602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 753/1506] 10.222.0.74 () {52 vars in 829 bytes} [Thu Jun 16 16:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:36:41.976728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:42.042261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9239b00-2772-4efd-af94-a94bd0c3ca88 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:42.046874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:42.047794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:42.048137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:42.048410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] PATH_INFO: `/v3/users/c6015e204332414b946abf9e58754342` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:42.049639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6015e204332414b946abf9e58754342)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:42.050670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:42.050670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:36:42.050670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:36:42.051898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:42.123633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6015e204332414b946abf9e58754342'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.124407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6015e204332414b946abf9e58754342'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.142041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6015e204332414b946abf9e58754342'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.164877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6015e204332414b946abf9e58754342'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.169352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:42.176918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6015e204332414b946abf9e58754342'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.184746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 754/1507] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:36:41 2022] DELETE /identity/v3/users/c6015e204332414b946abf9e58754342 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:36:42.189501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:42.190017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:42.190158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:42.209604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:42.210851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:42.273103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 754/1508] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:36:42.283016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:42.284582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:42.340912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b71e8-8651-49d0-8138-c608aaa10881 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:42.344685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:42.345339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:42.345624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:42.345899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:42.355384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:42.356857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366cc77-2e49-4a62-af27-59fdc294409b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:42.366378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 755/1509] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:36:42.717256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:42.718923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:42.731404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9404d62-7acc-4826-996c-1d3dbedc4586 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:42.737704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:42.738662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:42.739032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:42.739446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] PATH_INFO: `/v3/projects/b45e991e94644fa594765946422204b6` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:42.740791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45e991e94644fa594765946422204b6)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:42.742064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:36:42.742064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:36:42.742064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:36:42.743587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:42.812634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45e991e94644fa594765946422204b6', 'request_id': 'req-095b1142-f046-4e07-a1a2-bc54cdc72972'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.819215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45e991e94644fa594765946422204b6', 'request_id': 'req-095b1142-f046-4e07-a1a2-bc54cdc72972'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.829292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45e991e94644fa594765946422204b6', 'request_id': 'req-095b1142-f046-4e07-a1a2-bc54cdc72972'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.849049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] The token cache is being invalidate because project b45e991e94644fa594765946422204b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:36:42.849247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.849997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-095b1142-f046-4e07-a1a2-bc54cdc72972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:36:42.851586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 755/1510] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:36:42 2022] DELETE /identity/v3/projects/b45e991e94644fa594765946422204b6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:36:52.362215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:52.845934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:38:29.468726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65056db4-431c-450a-b793-ef8494c78080 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:29.469343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65056db4-431c-450a-b793-ef8494c78080 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:29.469343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65056db4-431c-450a-b793-ef8494c78080 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:29.470678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:29.477700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d97465-3118-450a-aef9-58be2958cd6a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:29.478488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d97465-3118-450a-aef9-58be2958cd6a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:29.478800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d97465-3118-450a-aef9-58be2958cd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:29.502979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d1d97465-3118-450a-aef9-58be2958cd6a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:29.504319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d1d97465-3118-450a-aef9-58be2958cd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:29.567396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:29.736233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65056db4-431c-450a-b793-ef8494c78080 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:29.739467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-65056db4-431c-450a-b793-ef8494c78080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:29.825272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65056db4-431c-450a-b793-ef8494c78080 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:29.831222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:29.831894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:29.832144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:29.832451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:29.840898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:29.842549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237381f-f446-4ed4-b1f0-d4590966eb5f service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:29.852480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 757/1513] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:30.723581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0eb0-0ab7-47ad-8229-f8b1beffec2d None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:30.724228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0eb0-0ab7-47ad-8229-f8b1beffec2d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:30.724228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0eb0-0ab7-47ad-8229-f8b1beffec2d None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:30.726297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:30.733657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae9f65-fed3-400e-84f1-9b994abb4cf7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:30.734156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae9f65-fed3-400e-84f1-9b994abb4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:30.734270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae9f65-fed3-400e-84f1-9b994abb4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:30.753913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7fae9f65-fed3-400e-84f1-9b994abb4cf7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:30.755142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7fae9f65-fed3-400e-84f1-9b994abb4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:30.762975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:38:30.921040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0eb0-0ab7-47ad-8229-f8b1beffec2d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:30.940489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0eb0-0ab7-47ad-8229-f8b1beffec2d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:30.947324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:30.947965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:30.948269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:30.948534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:30.957533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:30.959240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea4e0-734b-4a8a-afa2-9cb3c21ca7d5 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:30.969494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 758/1516] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:32.622775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec3ede-e832-4662-8302-6b5c4d326f0f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:32.623381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec3ede-e832-4662-8302-6b5c4d326f0f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:32.623381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec3ede-e832-4662-8302-6b5c4d326f0f None None] PATH_INFO: `/v3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:32.624698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 759/1517] 10.222.0.43 () {54 vars in 925 bytes} [Thu Jun 16 16:38:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:38:32.632548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9ed41-ef84-49f2-bd2a-3b5c91c616a4 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:32.632998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9ed41-ef84-49f2-bd2a-3b5c91c616a4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:32.633103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9ed41-ef84-49f2-bd2a-3b5c91c616a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:32.657697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5fe9ed41-ef84-49f2-bd2a-3b5c91c616a4 None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:32.658997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5fe9ed41-ef84-49f2-bd2a-3b5c91c616a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:32.739356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 759/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 16 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:40.747967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=37 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:38:42.309277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdc60-a85a-4942-b45f-e53f55a4740a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.309981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdc60-a85a-4942-b45f-e53f55a4740a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.309981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdc60-a85a-4942-b45f-e53f55a4740a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.311213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 760/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:42.316455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56747795-8447-4183-9a4a-24d1da8201d3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.316907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56747795-8447-4183-9a4a-24d1da8201d3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.317005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56747795-8447-4183-9a4a-24d1da8201d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.336625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-56747795-8447-4183-9a4a-24d1da8201d3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:42.337858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-56747795-8447-4183-9a4a-24d1da8201d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:42.345962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 760/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:42.355646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911bef9-c73c-42ec-be3a-4c6893822db3 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.355958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911bef9-c73c-42ec-be3a-4c6893822db3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.356235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911bef9-c73c-42ec-be3a-4c6893822db3 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.357706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 761/1521] 10.222.0.74 () {54 vars in 960 bytes} [Thu Jun 16 16:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:42.362568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.362862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.363183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.381362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:42.382706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:42.391098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 761/1522] 10.222.0.74 () {58 vars in 1057 bytes} [Thu Jun 16 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:42.399197 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911bef9-c73c-42ec-be3a-4c6893822db3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:42.412880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911bef9-c73c-42ec-be3a-4c6893822db3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:42.418064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:42.418710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.418997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.419313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.428069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:42.430076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d448f13-081c-4e98-aa74-e4111242168e service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:42.440309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 762/1523] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:42.499886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:42.513116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc17cdd-52f4-4755-8c1e-29278d99950f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:42.519180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:42.519851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:42.520137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:42.520389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:42.529606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:42.531233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c62541-fad4-4880-b1c1-00898d36afb4 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:42.541610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 762/1524] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:43.413544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db601-de38-4523-afaa-7f4b259efd9d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:43.414446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db601-de38-4523-afaa-7f4b259efd9d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:43.414878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db601-de38-4523-afaa-7f4b259efd9d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:43.417070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:43.422867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302737-a625-417d-aebf-d18ef6a87757 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:43.423600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302737-a625-417d-aebf-d18ef6a87757 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:43.423775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302737-a625-417d-aebf-d18ef6a87757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:43.445135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-93302737-a625-417d-aebf-d18ef6a87757 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:43.446359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-93302737-a625-417d-aebf-d18ef6a87757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:43.456532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:43.606628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db601-de38-4523-afaa-7f4b259efd9d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:43.632044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db601-de38-4523-afaa-7f4b259efd9d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:43.639332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:43.639984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:43.640267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:43.640541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:43.650882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:43.652492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eeeb9-99f4-4a4e-9b85-07f4d9785aa5 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:43.663184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 764/1527] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:48.103084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:48.103694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:48.103694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:48.105098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:48.113422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225352a-7995-4b37-9f67-04b47aa96748 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:48.114130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225352a-7995-4b37-9f67-04b47aa96748 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:48.114266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225352a-7995-4b37-9f67-04b47aa96748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:48.138254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8225352a-7995-4b37-9f67-04b47aa96748 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:48.139481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8225352a-7995-4b37-9f67-04b47aa96748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:48.150526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:48.187110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:48.189116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:48.252343 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c926e-c11a-4bef-9992-0335bd148a6e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:48.256641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:48.257362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:48.257544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:48.257941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:48.265066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:38:48.269294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:48.270782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34dba0d-79e9-430a-8ca1-ad289559e6e6 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:48.281557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 765/1530] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:50.394958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c23d6-c155-4450-b723-5ba5bb917b76 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:50.395846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c23d6-c155-4450-b723-5ba5bb917b76 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:50.395846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c23d6-c155-4450-b723-5ba5bb917b76 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:50.397868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:38:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:50.404289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9084-a4c8-421c-b2eb-fbc62b0b2015 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:50.404755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9084-a4c8-421c-b2eb-fbc62b0b2015 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:50.404908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9084-a4c8-421c-b2eb-fbc62b0b2015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:50.430163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2cac9084-a4c8-421c-b2eb-fbc62b0b2015 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:50.431028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2cac9084-a4c8-421c-b2eb-fbc62b0b2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:50.439307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:50.478688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c23d6-c155-4450-b723-5ba5bb917b76 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:50.492511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c23d6-c155-4450-b723-5ba5bb917b76 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:50.497396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:50.498029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:50.498273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:50.498583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:50.508156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:50.509596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821561ab-96b3-44bc-8527-e6ee50a95e8c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:50.519041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 767/1533] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:54.273619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:54.274237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:54.274237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:54.275645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:54.282126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63897be-9213-422d-b114-e660bf1c884f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:54.282913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63897be-9213-422d-b114-e660bf1c884f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:54.283113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63897be-9213-422d-b114-e660bf1c884f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:54.310932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f63897be-9213-422d-b114-e660bf1c884f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:54.312209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f63897be-9213-422d-b114-e660bf1c884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:54.321844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:54.588547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:54.591917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:54.656016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a78f-34f8-4192-aff5-32bb2c8750d0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:54.660091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:54.660736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:54.661020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:54.661297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:54.671011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:54.672477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe0f5-d4bc-483c-bc24-23b9a863746f service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:54.682052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 768/1536] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:38:56.820637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:56.821633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:56.821633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:56.822531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:38:56.829114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a41549-6506-4dbb-8cef-66133a20f45c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:56.829555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a41549-6506-4dbb-8cef-66133a20f45c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:56.829760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a41549-6506-4dbb-8cef-66133a20f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:56.856535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-74a41549-6506-4dbb-8cef-66133a20f45c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:38:56.857751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-74a41549-6506-4dbb-8cef-66133a20f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:56.866172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:38:57.129076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:57.131324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:38:57.146129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641115cf-eb24-4da3-bb31-f5de7cd275f2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:57.150173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:57.150813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:57.151138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:57.151410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:57.160710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:57.162149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a5c69-cdb1-4d35-88fe-3405371e5af3 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:57.180663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 770/1539] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:39:06.850486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=35 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:39:07.167982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=30 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:28.350370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.351922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.352442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.377983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.379642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:28.391836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 770/1540] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:28.402368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.403478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.404054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.436463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.437730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:28.446819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 771/1541] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:28.452052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.467686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4669f72-1da4-4329-ad0a-0617aabcb28a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.475409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.476119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.476446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.476719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.477848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:28.478172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.479902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.500803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 771/1542] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:28.509558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.524527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92171b41-4d3f-4230-b857-086d64d96f68 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.529572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.530255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.530538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.530815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.531552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.533668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee119b7-f7e3-4cf9-9b67-46a31030dfe3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.580103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 772/1543] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:28.584295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.603081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d478ee6-e863-4be7-b49b-a1670612b4f8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.693397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83230-dc6d-49c9-8c89-4aabf9690d9e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.693734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83230-dc6d-49c9-8c89-4aabf9690d9e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.693966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83230-dc6d-49c9-8c89-4aabf9690d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.695018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.695998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.696482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.696956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.697886 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.700316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.732007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d4e83230-dc6d-49c9-8c89-4aabf9690d9e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.733257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d4e83230-dc6d-49c9-8c89-4aabf9690d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:28.735225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 772/1544] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:28.739622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.742609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 773/1545] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:28.751844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0ee8-8f99-4816-bf30-9d0d55c97787 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.752773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.753108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.753424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.772169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.773277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.773787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.774297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.775300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.777320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.788547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.789923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:28.796884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 773/1546] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:40:28.803764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 774/1547] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:28.804735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.805057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.818383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076420-9a99-446d-8e27-29b3d78b3cd7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.821590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fce4ba-9ab8-40ff-b057-fd6f3639f284 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.826688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.827482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.827769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.828056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.829669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:28.829983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.835115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.835670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:28.835670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:40:28.835670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:40:28.836596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.836717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.837197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.837735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.838704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.840689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8ba06-e8bd-4c95-ab09-0c890f6ec96c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.853359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 774/1548] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:28.860004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 775/1549] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:28.860635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.861042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.861633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.862143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.878105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f81e0-6570-48a0-a1e4-0a61ddc6664c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.883843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:28.884775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.885056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.885325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] PATH_INFO: `/v3/projects/9b80d169e3124b04b4a2f31017ad6eb3/users/90f6f184c562423ea57295465ec5b211/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.904295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.905594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:28.906601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b80d169e3124b04b4a2f31017ad6eb3, user_id=90f6f184c562423ea57295465ec5b211, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:28.909406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591fa4c-2b2b-4abb-9fa3-2d63444ae4bd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:28.922753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 775/1550] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:28.926000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:28.931448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 776/1551] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:28 2022] PUT /identity/v3/projects/9b80d169e3124b04b4a2f31017ad6eb3/users/90f6f184c562423ea57295465ec5b211/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:28.933253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cf9e0-4fae-4e2b-a83b-6fdd9b0773e0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:28.933542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cf9e0-4fae-4e2b-a83b-6fdd9b0773e0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:28.933809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cf9e0-4fae-4e2b-a83b-6fdd9b0773e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:28.947414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1929d4c-b9f9-4eee-a7a8-2e577737e349 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:28.957800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-457cf9e0-4fae-4e2b-a83b-6fdd9b0773e0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:28.959044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-457cf9e0-4fae-4e2b-a83b-6fdd9b0773e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.018519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.019215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.019501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.019774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.020460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.021478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:29.021478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:40:29.021478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:40:29.022669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5a1bb-f0d4-44eb-9df5-d61f865c2c76 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.033404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 777/1552] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.034778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce197-4f7b-44ab-9c4c-71d30c09031a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.035088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce197-4f7b-44ab-9c4c-71d30c09031a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.035367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecce197-4f7b-44ab-9c4c-71d30c09031a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.054626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 776/1553] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:28 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.054626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f50f3-24e2-40ec-9ab1-c2cbbbd30578 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.054626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f50f3-24e2-40ec-9ab1-c2cbbbd30578 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.054626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f50f3-24e2-40ec-9ab1-c2cbbbd30578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.059627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4ecce197-4f7b-44ab-9c4c-71d30c09031a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.060867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4ecce197-4f7b-44ab-9c4c-71d30c09031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.072149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-608f50f3-24e2-40ec-9ab1-c2cbbbd30578 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.073388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-608f50f3-24e2-40ec-9ab1-c2cbbbd30578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.094901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.121073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 778/1554] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.122855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30e1bb-21d4-4020-ba06-c3728d661cfa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.123160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30e1bb-21d4-4020-ba06-c3728d661cfa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.123439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30e1bb-21d4-4020-ba06-c3728d661cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.137528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 777/1555] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.138844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.139174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.139458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.146469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce30e1bb-21d4-4020-ba06-c3728d661cfa None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.147713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce30e1bb-21d4-4020-ba06-c3728d661cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.156557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 779/1556] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.157994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfc79-147b-4024-9b62-392ecceb7839 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.158287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfc79-147b-4024-9b62-392ecceb7839 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.158744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dfc79-147b-4024-9b62-392ecceb7839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.170735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] MFA Rules not processed for user `90f6f184c562423ea57295465ec5b211`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.172085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.176682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7c2dfc79-147b-4024-9b62-392ecceb7839 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.177856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7c2dfc79-147b-4024-9b62-392ecceb7839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.186104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 780/1557] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.192993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984722f-80c7-4344-8a9f-8257dcf232f0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.193487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984722f-80c7-4344-8a9f-8257dcf232f0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.194522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984722f-80c7-4344-8a9f-8257dcf232f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.214682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b984722f-80c7-4344-8a9f-8257dcf232f0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.216595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b984722f-80c7-4344-8a9f-8257dcf232f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.225207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 781/1558] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.226553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a645e2-2e03-4680-9e2f-fe47d64c6b9b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.226888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a645e2-2e03-4680-9e2f-fe47d64c6b9b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.227486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a645e2-2e03-4680-9e2f-fe47d64c6b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.234135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.247562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 778/1559] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.248815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.252096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-29a645e2-2e03-4680-9e2f-fe47d64c6b9b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.253333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-29a645e2-2e03-4680-9e2f-fe47d64c6b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.259625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0731f67-e3bb-469f-ba25-6889ebf816a6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.261928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 782/1560] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.264466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.264622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.264873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.265053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.265150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.265428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.265772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.266468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.268585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74175-f346-43d5-a8cf-076746fccb6f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.283360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.284742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.292211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 779/1561] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.293518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634497-d31a-45eb-ac5f-be4f80ddb126 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.293806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634497-d31a-45eb-ac5f-be4f80ddb126 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.294300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634497-d31a-45eb-ac5f-be4f80ddb126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.294424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 783/1562] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.295983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.306602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2af9-f718-4a76-815b-d6b3ac55a3f1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.311598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.312312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.312609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.312889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.313482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1a634497-d31a-45eb-ac5f-be4f80ddb126 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.313639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.313964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.314708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1a634497-d31a-45eb-ac5f-be4f80ddb126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.315622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3985a83-da84-4d79-871b-d56deeec6139 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.322768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 780/1563] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.324161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.324458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.324777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.328425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 784/1564] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.329917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.330209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.330488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.348052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.348232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:29.349200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.349339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:29.357268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 785/1565] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.357419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 781/1566] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:29.358308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.362105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.367806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482eee51-df55-4e60-910f-9a16792e2210 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.371824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fabf6-74ac-4030-b8c4-72a811c16f2f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.372364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.373051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.373331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.373603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.374271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.376271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.376820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.377512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.377762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.378042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.379181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.379497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.381153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.393300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 782/1567] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.394244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.406191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005d2a9-e4e5-43bd-a9ec-f06bcfd4b3c2 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.406604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 786/1568] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.407477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.467732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.468413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.468705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.468981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.469718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.471398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.471697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b731c-6862-45cc-a60e-d3abb9653d19 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.476361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.477036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.477329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.477599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.478381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.478692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.480331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.481893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 783/1569] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:29.483060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.492541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 787/1570] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.492742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697ae4-475e-446e-bf20-ffffe64f1ec4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.493429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.499143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.499934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.500064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.500314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.501374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.501690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.503313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.503518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ce899-3635-4748-b2b6-454224a8577a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.506371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.509059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.509765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.510048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.510322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.510996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.513067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.525289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 784/1571] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.525289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.536841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180cfa-d2aa-4de1-98bc-9053220a2bf0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.542809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.547166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.547166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.547166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.547166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.547166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.547669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 788/1572] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.547669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.547851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.562086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 785/1573] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.563038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.572885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5741792-6b66-43b3-956c-3759bb417e1f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.615987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144fcea-22eb-4433-a4d3-1078d98e3406 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.620567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.621239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.621560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.621841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.622768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:29.623111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.624775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.630342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.631123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.631493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.631843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.632605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.634614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.636687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 789/1574] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:29.637664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.649540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12112-7b87-4387-b882-e64709fcb31d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.654751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.655454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.655744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.656031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.656723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.657993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:29.657993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:40:29.657993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:40:29.659243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.666680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 786/1575] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.666680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.669528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.684741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 790/1576] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.684741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.692260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c3-864d-4a89-b173-8d2061b63536 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.730765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b71b-11f8-44dc-b108-2f46b963c375 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.735489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.736170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.736483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.736759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.737500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.739185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.749786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 787/1577] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:29.751962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.753219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.754427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.754784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.755092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.755807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.757738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.762533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439ea94-525e-4143-9bfe-ef9880148464 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.767520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.768191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.768475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.768751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.769416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.771336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.787093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 791/1578] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.787314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.801262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 788/1579] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.802190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.848100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.849953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64542e-3ab4-404d-a522-2e0283f9ddcd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.866925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85aab8-d50a-456d-8308-f9bc7c8b7d1d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.875503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.876476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.876778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.877061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.877821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.881032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.911768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 789/1580] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.911768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.913856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.914569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.914841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.915138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.916244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.917803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:29.941544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 792/1581] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:29.942621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:29.952207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b15f-c09a-4c2e-ba0b-4ebf258c43f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.964638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:29.977773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e0374-8824-47e2-aa85-0dcefba4c099 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:29.982448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:29.983137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:29.983431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:29.983707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:29.984435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:29.986369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.012622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.013424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.013707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.014192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.014316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 790/1582] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.014736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.014873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.016723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.037395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 793/1583] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:29 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.041391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.051418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3689-5b8a-4d1d-acd2-72bdb3dd96d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.084281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0602e-19cf-4a9e-8fd9-cb229a0a095c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.089600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.090482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.090873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.091265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.092070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.093763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.103914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 791/1584] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.104767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.110737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.111744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.112180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.112308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] PATH_INFO: `/v3/projects/44c2d91332524685bb77468a828daa41/users/c2ebf15071c84bec9126894621f23387/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.116875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf68ee9-76c4-4862-9a10-88b7fe69adce admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.121377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.122021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.122306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.122577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.123360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.125258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.127333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c2d91332524685bb77468a828daa41, user_id=c2ebf15071c84bec9126894621f23387, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.130366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.145368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 794/1585] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/44c2d91332524685bb77468a828daa41/users/c2ebf15071c84bec9126894621f23387/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.146329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.151742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 792/1586] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.152652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.164849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3232ef-7927-4419-b4ea-6365073bc8c8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.196486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:30.219172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b216c-f186-4687-919c-b9d95c40c1a9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.223994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.224664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.224946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.225221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.225932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.227486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.227885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.228169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.228455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.228731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.229429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.231445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.251301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 795/1587] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.251301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.254735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 793/1588] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.254735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.260572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763a762-f76a-4cdb-b18b-85aea41995a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.262728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cac5-9cac-4466-94a3-16caf444865f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.265463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.266158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.266449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.266728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.267244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.267492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.267907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.268194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.268475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.269120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.269242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.271095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.289162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 796/1589] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.290070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.294392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 794/1590] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.294392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.301940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb60-b457-4b8e-8997-9bbeb94f7b08 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.304434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9277a-cd45-4994-b2f1-347e457661b8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.307220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.307924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.308185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.308463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.309013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.309889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.309997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.310176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.310454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] PATH_INFO: `/v3/projects/fe123b832379452d89993eb64bc4bbab/users/3b69f3e0a7f348d29b0290098714190d/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.311615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.313934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:30.323956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 797/1591] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.325351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.328150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe123b832379452d89993eb64bc4bbab, user_id=3b69f3e0a7f348d29b0290098714190d, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.329212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:30.329212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:40:30.329212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:40:30.331168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7146f-1fb9-43ed-ab88-b46dfa670817 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.335344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16a2-6f30-47cb-a0a0-a99e224c56f1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.340515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.341172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.341460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.341740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.342512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.344150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.345602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 795/1592] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/fe123b832379452d89993eb64bc4bbab/users/3b69f3e0a7f348d29b0290098714190d/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.346996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.347316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.347594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.357196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 798/1593] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.358216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.368369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a925-ef58-4d58-a0e7-127f62b2408b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.377588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] MFA Rules not processed for user `c2ebf15071c84bec9126894621f23387`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:30.379123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:30.428023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.428700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.428994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.429272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.430050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.431734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.441804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 799/1594] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.442691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.449278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 796/1595] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:30.450306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.453070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72da02-5aec-4abe-ba9c-0b94e2b47f8c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.457999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.458676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.458962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.459272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.460018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.460776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b4880-1453-4c20-8c72-d9151d7aa9e3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.461655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.465395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.466079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.466369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.466647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.467512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.469209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.472639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 800/1596] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.473619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.483466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 797/1597] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.483632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e09a4-35a2-4eb3-9590-b7d58617fdfa admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.484142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.490621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.491530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.491809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.492078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.492811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.494441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.495611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a7304-69d3-4898-8b2b-c606e5e88214 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.500512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.501187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.501471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.501748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.502505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.504450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.505127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 801/1598] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.506023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.515697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b2c5-2385-4964-a4fc-12ce352efcbe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.517926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 798/1599] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.518574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.523551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.524262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.524534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.524812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.525582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.527374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.531333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc6145-9943-4e10-a292-e7dd7b5476b5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.536548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.537321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.537721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.537999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.538735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.540384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.541900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 802/1600] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.542760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.553531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcaa7b0-1ba2-41ea-a217-dea9dc7e4c3c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.554071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 799/1601] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.554988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.558575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.559334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.559627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.560054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.560800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.562436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.565430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcfb2f-5dbd-45a4-876c-d126d602417f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.570994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.571685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.571969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.572237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.572913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.576794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 803/1602] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.576794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.577076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.586219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9d06d-d8a5-4e38-9168-f49e9b0a6641 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.591078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.591752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.592040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.592321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.593128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.594959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.607511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 800/1603] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.607511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.607782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 804/1604] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.607782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.617462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0129-fb94-40eb-91cd-1ea40b40e9ac admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.621597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:30.675962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.676191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6f14-89a1-4046-bac3-fbb9b14f072e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.676537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.676830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.677128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.677944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.679583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.681265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.682516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.682798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.683073 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] PATH_INFO: `/v3/projects/627133a3363742fda58273b067fab968/users/80113e82c4a74f4e8fd42b88aca3920d/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.693092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 805/1605] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:30.693092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.700770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627133a3363742fda58273b067fab968, user_id=80113e82c4a74f4e8fd42b88aca3920d, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.703605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c953a3-756a-4a95-afdc-dddbe10f1bae admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.704482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.708335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.709334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.709696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.709990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] PATH_INFO: `/v3/projects/8725edcf74cc463cafe3614a3bd880e6/users/b49313559188427bb9f42068583a8292/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.719694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 801/1606] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/627133a3363742fda58273b067fab968/users/80113e82c4a74f4e8fd42b88aca3920d/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.720492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.730645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8725edcf74cc463cafe3614a3bd880e6, user_id=b49313559188427bb9f42068583a8292, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.733516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.743773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:30.755009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 806/1607] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/8725edcf74cc463cafe3614a3bd880e6/users/b49313559188427bb9f42068583a8292/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.755885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.784783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340cf7-f754-41e1-b954-020ba81f7815 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.821214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3896cb8-c599-499e-ad86-209794dd16bd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.826242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.827251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.827536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.827816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] PATH_INFO: `/v3/projects/e136d4cbac6e44a59aebfc722a03b7e6/users/d9d03b65829f4e1d8e497c6c30ccfcdf/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.843295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e136d4cbac6e44a59aebfc722a03b7e6, user_id=d9d03b65829f4e1d8e497c6c30ccfcdf, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.844549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.845527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.845811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.846027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.846150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] PATH_INFO: `/v3/projects/fe123b832379452d89993eb64bc4bbab/users/3b69f3e0a7f348d29b0290098714190d/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.848186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe123b832379452d89993eb64bc4bbab, user_id=3b69f3e0a7f348d29b0290098714190d, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.850800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.861796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 807/1608] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/e136d4cbac6e44a59aebfc722a03b7e6/users/d9d03b65829f4e1d8e497c6c30ccfcdf/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.862626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 802/1609] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/fe123b832379452d89993eb64bc4bbab/users/3b69f3e0a7f348d29b0290098714190d/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.862746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.863897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.915183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:30.937198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e284c-93c5-421f-96b3-19ea83c86808 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.938486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb59e4-7433-421a-acde-de793e0154fa admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:30.941953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.942856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.943122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:30.943311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.943829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] PATH_INFO: `/v3/projects/28061579532249b7922ab14c12529580/users/556cbb2920b448dc81bc90606cfc59f4/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.943952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:30.944224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:30.944507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:30.945200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.947266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.960183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28061579532249b7922ab14c12529580, user_id=556cbb2920b448dc81bc90606cfc59f4, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:30.962993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:30.969903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 803/1610] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:30 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:30.970788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.976981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 808/1611] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/28061579532249b7922ab14c12529580/users/556cbb2920b448dc81bc90606cfc59f4/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:30.978122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:30.982870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba2c49-5e11-4929-93e1-638b76128c3a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.047524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.047989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.048273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.048551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] PATH_INFO: `/v3/projects/17861fed91df41c7916929aef6783c45/users/79f1550993cd4952896cbf4813a76233/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.051043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1b17-db62-4105-b972-969a0e40762c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.055842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.056760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.057050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.057361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] PATH_INFO: `/v3/projects/0a7a93f8b61c48759abc98e8b86ad4e8/users/5ab5b939a13e43469b3a4a48da559c83/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.064504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17861fed91df41c7916929aef6783c45, user_id=79f1550993cd4952896cbf4813a76233, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.067537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205853fd-1e71-42b5-884f-858e49172dd1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.069978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:31.074090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7a93f8b61c48759abc98e8b86ad4e8, user_id=5ab5b939a13e43469b3a4a48da559c83, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.077826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1533deb-ca00-45d8-b231-b58b6a9e3b00 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.085912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 804/1612] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/17861fed91df41c7916929aef6783c45/users/79f1550993cd4952896cbf4813a76233/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:31.087221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690ac9-1148-4123-98d0-b86d21f5351c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.087646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690ac9-1148-4123-98d0-b86d21f5351c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.087934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690ac9-1148-4123-98d0-b86d21f5351c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.096579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 809/1613] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:30 2022] PUT /identity/v3/projects/0a7a93f8b61c48759abc98e8b86ad4e8/users/5ab5b939a13e43469b3a4a48da559c83/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:31.097855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6093f1-c587-4b16-8a11-c934071df5e6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.098170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6093f1-c587-4b16-8a11-c934071df5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.098422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6093f1-c587-4b16-8a11-c934071df5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.112362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c9690ac9-1148-4123-98d0-b86d21f5351c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.113629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c9690ac9-1148-4123-98d0-b86d21f5351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.116357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4e6093f1-c587-4b16-8a11-c934071df5e6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.117490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4e6093f1-c587-4b16-8a11-c934071df5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.173372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 805/1614] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.174707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.174994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.175305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.177958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 810/1615] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.181130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40fc44-3d46-43de-b597-fd6fbffa7adf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.181130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40fc44-3d46-43de-b597-fd6fbffa7adf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.181130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40fc44-3d46-43de-b597-fd6fbffa7adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.205882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] MFA Rules not processed for user `d9d03b65829f4e1d8e497c6c30ccfcdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.207093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.209305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ca40fc44-3d46-43de-b597-fd6fbffa7adf None None] MFA Rules not processed for user `3b69f3e0a7f348d29b0290098714190d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.210505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ca40fc44-3d46-43de-b597-fd6fbffa7adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.250117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:31.275957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 806/1616] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.276820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.284896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 811/1617] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.288426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c057ac-f030-485b-ac34-2c7434808c0a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.288735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c057ac-f030-485b-ac34-2c7434808c0a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.288999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c057ac-f030-485b-ac34-2c7434808c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.291752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1c6e-0ed0-44c2-8ac1-9a92c8cc55b8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.297150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.297837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.298125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.298406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.299193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.300820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05e76-54f6-4e1f-b5ee-9dc0929a3b1a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.311672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 807/1618] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:31.312912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.313202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.313483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.318084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-06c057ac-f030-485b-ac34-2c7434808c0a None None] MFA Rules not processed for user `556cbb2920b448dc81bc90606cfc59f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.319334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-06c057ac-f030-485b-ac34-2c7434808c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.341991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] MFA Rules not processed for user `79f1550993cd4952896cbf4813a76233`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.343669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.394110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 812/1619] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.395590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.395926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.396216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.402614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:31.416332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.417584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.422003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 808/1620] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.422859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.425742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 813/1621] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.426558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.426674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.484166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7d082-8aa3-4a4c-a5a2-1e88c7a109f5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.484845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25613ca-8f0f-4f19-a342-728807c85b3f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.487824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.487975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.488499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.488608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.488799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.488890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.489057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.489208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.498437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.499321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.499957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.500862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.509842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 814/1622] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:31.510625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.521688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 809/1623] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:31.522990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.544978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0a1d0-bfdb-4273-b1f8-29542317bb3c service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.558975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.560257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.560623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.561101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] PATH_INFO: `/v3/projects/402fa8d4c9224ad8a5028295e6a90483/users/d02d5f743bbc4c53b4e759a87423ea86/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.574376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9020580-b406-4aa0-a525-9199a9e51d3e service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.578891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.579492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402fa8d4c9224ad8a5028295e6a90483, user_id=d02d5f743bbc4c53b4e759a87423ea86, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.580205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.580648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.581084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.582931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e645c-f7e7-4fd3-8171-90c7521d49b6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.594894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.596771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.602096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 810/1624] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:31 2022] PUT /identity/v3/projects/402fa8d4c9224ad8a5028295e6a90483/users/d02d5f743bbc4c53b4e759a87423ea86/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:31.604538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c53d14-cff4-4148-9573-155c9ab7e0a0 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.605692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c53d14-cff4-4148-9573-155c9ab7e0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.606195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c53d14-cff4-4148-9573-155c9ab7e0a0 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.608467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 811/1625] 10.222.0.74 () {54 vars in 961 bytes} [Thu Jun 16 16:40:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:31.610258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d65a-d727-49db-a57e-39b90c860141 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.610647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d65a-d727-49db-a57e-39b90c860141 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.611124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d65a-d727-49db-a57e-39b90c860141 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.613190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 812/1626] 10.222.0.74 () {54 vars in 961 bytes} [Thu Jun 16 16:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:31.614122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d65a-d727-49db-a57e-39b90c860141 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.620496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 815/1627] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:31.622092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.632870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0476335-2d61-4696-89bc-8c23cb5a7a55 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.695181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:31.719643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.720631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.721065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.721441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.740707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d65a-d727-49db-a57e-39b90c860141 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.748831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.757482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.757482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.758369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.760738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.763116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.794840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 813/1628] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/projects => generated 403 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:31.800710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:31.806392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:31.821833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.824751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b8577-5ec8-4602-bcf4-e49376672675 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.839377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 816/1629] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:31.841361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.841721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.842091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.882060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:31.884592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:31.939428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0d6e3-1a6a-45b9-a0a4-91acb29c9c85 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:31.955175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:31.956008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.956499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.956849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.957713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:31.961534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b428867-61cd-4773-9ef1-51c3ec9af3d7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:31.992615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 814/1630] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:31.994228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:31.994372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 817/1631] 10.222.0.74 () {58 vars in 1058 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:31.994551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:31.994828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:31.996282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.035426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:32.036830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:32.063290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 815/1632] 10.222.0.74 () {58 vars in 1058 bytes} [Thu Jun 16 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:32.064383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.083551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb302-6a96-4496-ada8-531ce79d5865 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.091746 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.093165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.093635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.094082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.094970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.097817 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.113293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57040d-67ac-4829-80f9-0a55c7ed06e2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.118445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.119455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.119907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.120335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.132046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 816/1633] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:32 2022] POST /identity/v3/users => generated 308 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:32.133582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.148282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b4ea-0ce7-4a69-960b-f902bf93dfa4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.154656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.156625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.157735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.158327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] PATH_INFO: `/v3/projects/402fa8d4c9224ad8a5028295e6a90483/users/d02d5f743bbc4c53b4e759a87423ea86/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.160974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402fa8d4c9224ad8a5028295e6a90483, user_id=d02d5f743bbc4c53b4e759a87423ea86, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.164643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.190024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 817/1634] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:32 2022] PUT /identity/v3/projects/402fa8d4c9224ad8a5028295e6a90483/users/d02d5f743bbc4c53b4e759a87423ea86/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:32.193341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.230090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.235011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.254310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 818/1635] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:32.255562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.275128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4bb0b-55eb-4440-841c-21c5fb211b91 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.372583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:32.406377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee49-a515-4351-adfa-8e42bbe031f6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.416322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.417050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.417335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.417724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.443464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.444276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.444571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.444856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.445485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:32.502182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.504291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.525828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 818/1636] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:32.528687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.589998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.595143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdff72b-4320-467b-84e1-c29f381ff8cd service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.619702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 819/1637] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:32.624495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.625799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.626077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.696409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] MFA Rules not processed for user `d02d5f743bbc4c53b4e759a87423ea86`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:32.697685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:32.752159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a326b7b-eb23-4104-bd75-130844451eed service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.768788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.769509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.773229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.773536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.774370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.775985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.820488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 819/1638] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:32.831938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.851595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393b41d-3646-4691-8285-98fa1f6cfeb6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.861178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 820/1639] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:32.867075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:32.870040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:32.870341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:32.870621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] PATH_INFO: `/v3/projects/7256f9eef6814690b8c94987358f51df/users/38c5c0b2921e42d0a51ebb6c9983de37/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:32.875087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.913495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7256f9eef6814690b8c94987358f51df, user_id=38c5c0b2921e42d0a51ebb6c9983de37, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:32.916338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:32.939555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 820/1640] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:32 2022] PUT /identity/v3/projects/7256f9eef6814690b8c94987358f51df/users/38c5c0b2921e42d0a51ebb6c9983de37/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:32.945316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:32.958375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b78e9-a6b9-4f98-9f68-f880d90fe2a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:32.976258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:33.024669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:33.025753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.026272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.026623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.093645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc06f2-9aad-4040-9909-a90116a95955 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:33.100771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:33.102123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.102603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.103157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.104058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:33.107347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17588f7-5b13-4704-ab4e-afa429640e39 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:33.145366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 821/1641] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:33.145555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:33.145555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6bf85-84ac-4057-9d14-e31a630646c7 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:33.147496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c54ffe-91c9-44fe-9538-642b2c22320e None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.150014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c54ffe-91c9-44fe-9538-642b2c22320e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.150479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c54ffe-91c9-44fe-9538-642b2c22320e None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.157557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 822/1642] 10.222.0.74 () {54 vars in 912 bytes} [Thu Jun 16 16:40:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:33.158740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 821/1643] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:33.163667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c54ffe-91c9-44fe-9538-642b2c22320e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:33.164363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7bd8-9c61-427a-8eb2-0399a473ca3a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.164865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7bd8-9c61-427a-8eb2-0399a473ca3a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.165234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7bd8-9c61-427a-8eb2-0399a473ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.192829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c54ffe-91c9-44fe-9538-642b2c22320e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:33.200639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-58cb7bd8-9c61-427a-8eb2-0399a473ca3a None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:33.203220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-58cb7bd8-9c61-427a-8eb2-0399a473ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:33.204931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:33.206020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.206317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.206864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] PATH_INFO: `/v3/projects/7256f9eef6814690b8c94987358f51df/users/38c5c0b2921e42d0a51ebb6c9983de37/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.210179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7256f9eef6814690b8c94987358f51df, user_id=38c5c0b2921e42d0a51ebb6c9983de37, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:33.213493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507796f7-88b1-43b9-a512-3547cbaa0638 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:33.247083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 823/1644] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:33 2022] PUT /identity/v3/projects/7256f9eef6814690b8c94987358f51df/users/38c5c0b2921e42d0a51ebb6c9983de37/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:33.247083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e41a-c86f-413a-b320-4b3758584a42 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.247083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e41a-c86f-413a-b320-4b3758584a42 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.247083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e41a-c86f-413a-b320-4b3758584a42 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.248681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 824/1645] 10.222.0.74 () {54 vars in 912 bytes} [Thu Jun 16 16:40:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:33.253005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.253005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.253005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.299567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:33.300948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:33.329984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:33.395936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 822/1646] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:33.397429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.398423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.398896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.472878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] MFA Rules not processed for user `38c5c0b2921e42d0a51ebb6c9983de37`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:33.476329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:33.477516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 825/1647] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:33.478562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:33.615252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8255fab-3ecc-4317-83fd-d05faeca8feb None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:33.629692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:33.632350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:33.634194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.634485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.634765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.673540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:33.676836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:33.701985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 823/1648] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:33.703919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:33.716404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 826/1649] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:33.723335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee33e2-a598-412d-802c-df64fabfb908 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:33.727586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:33.728297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:33.728586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:33.728862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:33.748738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:33.751033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:33.767132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 824/1650] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:34.960247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:34.977487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e5a0c-0c89-4e80-988c-db28ea2e7e37 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:34.987947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:34.988854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:34.990705 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:34.991298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:35.036773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:35.041586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bdd0e-083f-49ec-b1ce-92b9be52a6e3 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:35.070493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 827/1651] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:36.345405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:36.605213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebaa55-cec9-465d-b2b2-92ea4b043d45 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:36.625276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:36.626372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:36.626513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:36.626738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] PATH_INFO: `/v3/users/79f1550993cd4952896cbf4813a76233` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:36.631653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f1550993cd4952896cbf4813a76233)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:36.647819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:36.853860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f1550993cd4952896cbf4813a76233'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:36.860662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f1550993cd4952896cbf4813a76233'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:36.890647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f1550993cd4952896cbf4813a76233'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:36.939718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f1550993cd4952896cbf4813a76233'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:36.965096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f1550993cd4952896cbf4813a76233'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:37.031749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 825/1652] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:40:36 2022] DELETE /identity/v3/users/79f1550993cd4952896cbf4813a76233 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:37.038792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:37.038792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:37.038792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:37.080145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:37.089568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:37.340197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 828/1653] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:37.453139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:37.455364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:37.618372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d2b00-816d-42d5-94e5-bab37f4f6456 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:37.635909 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:37.638087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:37.638524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:37.641128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:37.663863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:37.672491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:37.675012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5035e4-6778-49eb-a33b-69cca9e39d41 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:37.689386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 826/1654] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:39.358308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:39.362027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:39.395268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86990ec9-bc18-4aae-81ce-5effe136024b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:39.412489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:39.414446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:39.414969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:39.415615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] PATH_INFO: `/v3/users/c2ebf15071c84bec9126894621f23387` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:39.417213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ebf15071c84bec9126894621f23387)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:39.418784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: WARNING py.warnings [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:39.418784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: warnings.warn(msg) Jun 16 16:40:39.418784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: Jun 16 16:40:39.420551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:39.531685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:39.600734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ebf15071c84bec9126894621f23387'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:39.601690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ebf15071c84bec9126894621f23387'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:39.622650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ebf15071c84bec9126894621f23387'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:39.666034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ebf15071c84bec9126894621f23387'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:39.718613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ebf15071c84bec9126894621f23387'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:39.735939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 829/1655] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:40:39 2022] DELETE /identity/v3/users/c2ebf15071c84bec9126894621f23387 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:39.741093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:39.741577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:39.741768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:39.771624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:39.772462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:39.935189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 827/1656] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:39.964515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:39.966062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:39.994841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:39.996682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:40.025850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d92a0-09eb-45cd-9fb0-4600ac972d89 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:40.049857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:40.050756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:40.051045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:40.051353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] PATH_INFO: `/v3/projects/17861fed91df41c7916929aef6783c45` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:40.056846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17861fed91df41c7916929aef6783c45)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:40.057927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:40:40.057927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jun 16 16:40:40.057927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: Jun 16 16:40:40.059378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:40.133900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b0bd1-1db6-4486-a106-3ce031a3b671 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:40.143855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:40.144566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:40.144869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:40.145161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:40.178106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:40.182731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:40.230097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 830/1657] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:40:40.248241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17861fed91df41c7916929aef6783c45', 'request_id': 'req-a8835f58-9d08-4527-8593-31918a284969'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.262611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17861fed91df41c7916929aef6783c45', 'request_id': 'req-a8835f58-9d08-4527-8593-31918a284969'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.297481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17861fed91df41c7916929aef6783c45', 'request_id': 'req-a8835f58-9d08-4527-8593-31918a284969'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.335898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] The token cache is being invalidate because project 17861fed91df41c7916929aef6783c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:40:40.336207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.336922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8835f58-9d08-4527-8593-31918a284969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.338826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 828/1658] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:40:39 2022] DELETE /identity/v3/projects/17861fed91df41c7916929aef6783c45 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:40.343040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:40.357904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568f9f1-4470-48cb-ad16-dde4d596b6f0 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:40.590247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:40.591764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:40.592125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:40.596009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] PATH_INFO: `/v3/users/38c5c0b2921e42d0a51ebb6c9983de37` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:40.598170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c5c0b2921e42d0a51ebb6c9983de37)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:40.605688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:40.767187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c5c0b2921e42d0a51ebb6c9983de37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.771487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c5c0b2921e42d0a51ebb6c9983de37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.780167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:40.789039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c5c0b2921e42d0a51ebb6c9983de37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.829512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c5c0b2921e42d0a51ebb6c9983de37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.872807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c5c0b2921e42d0a51ebb6c9983de37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:40.901902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 831/1659] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:40:40 2022] DELETE /identity/v3/users/38c5c0b2921e42d0a51ebb6c9983de37 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:41.170645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:41.171999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:41.172423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:41.228259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] MFA Rules not processed for user `b49313559188427bb9f42068583a8292`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:41.229876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:41.343835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:41.345350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:41.398793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 829/1660] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:41.417361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:41.419550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:41.436931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:41.571257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37bbb-1712-4b60-8369-2a9a0488839f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:41.580581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:41.581257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:41.581548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:41.581822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:41.601934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:41.603802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:41.608166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87126-9ea3-483a-800d-b747ccca0ee0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:41.622879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:41.624057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:41.624441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:41.624807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] PATH_INFO: `/v3/projects/44c2d91332524685bb77468a828daa41` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:41.627826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 830/1661] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:40:41.644113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c2d91332524685bb77468a828daa41)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:41.651145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:41.806051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c2d91332524685bb77468a828daa41', 'request_id': 'req-9549846a-d993-47a4-8cc5-d951715e5c79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:41.832685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c2d91332524685bb77468a828daa41', 'request_id': 'req-9549846a-d993-47a4-8cc5-d951715e5c79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:41.852181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c2d91332524685bb77468a828daa41', 'request_id': 'req-9549846a-d993-47a4-8cc5-d951715e5c79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:41.878716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] The token cache is being invalidate because project 44c2d91332524685bb77468a828daa41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:40:41.879769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:41.883105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:41.888253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 832/1662] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:40:41 2022] DELETE /identity/v3/projects/44c2d91332524685bb77468a828daa41 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:41.889804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:41.924741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30076c7-6c7b-4139-8c7c-35fa503dfabc service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:42.063531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:42.065770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:42.066134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:42.066521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] PATH_INFO: `/v3/users/d02d5f743bbc4c53b4e759a87423ea86` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:42.068270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02d5f743bbc4c53b4e759a87423ea86)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:42.071980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:42.158483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02d5f743bbc4c53b4e759a87423ea86'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.159455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02d5f743bbc4c53b4e759a87423ea86'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.181353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02d5f743bbc4c53b4e759a87423ea86'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.226513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02d5f743bbc4c53b4e759a87423ea86'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.246041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a12996a9-7a31-4adf-9044-03be3c9733db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02d5f743bbc4c53b4e759a87423ea86'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.258566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 831/1663] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:40:41 2022] DELETE /identity/v3/users/d02d5f743bbc4c53b4e759a87423ea86 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:42.656342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:42.658590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:42.680616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549846a-d993-47a4-8cc5-d951715e5c79 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:42.801020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:42.802306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:42.802723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:42.803187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] PATH_INFO: `/v3/projects/7256f9eef6814690b8c94987358f51df` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:42.811839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7256f9eef6814690b8c94987358f51df)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:42.814191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:42.817584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:42.935868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7256f9eef6814690b8c94987358f51df', 'request_id': 'req-f1a8e794-438c-4520-af8f-fd3c7f7e5881'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.943351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7256f9eef6814690b8c94987358f51df', 'request_id': 'req-f1a8e794-438c-4520-af8f-fd3c7f7e5881'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.955832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7256f9eef6814690b8c94987358f51df', 'request_id': 'req-f1a8e794-438c-4520-af8f-fd3c7f7e5881'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.984446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] The token cache is being invalidate because project 7256f9eef6814690b8c94987358f51df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:40:42.984446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.984865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f1a8e794-438c-4520-af8f-fd3c7f7e5881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:42.986665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 833/1664] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:40:42 2022] DELETE /identity/v3/projects/7256f9eef6814690b8c94987358f51df => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:42.992527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:42.992797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:42.993070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.049022 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:43.050834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:43.081250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:43.236746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 832/1665] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:43.244484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.244923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.245357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.281420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:43.294287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:43.302771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 834/1666] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:43.306363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.320039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4169c2-cf7f-451d-a3d0-a478a4ee5e2e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.335249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.336159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.336624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.338605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.339727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:43.340336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.350386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.398584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 833/1667] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:43.402825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.414246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3705a6-79d0-48dd-b570-309685fdcace None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.419342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.420058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.420352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.421375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.425780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.428307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.475869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 835/1668] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:43 2022] POST /identity/v3/projects => generated 407 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:43.480108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.493745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968db33-f8e6-4f32-a6ab-287504f39012 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.617920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.620003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:43.649806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.650576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.650858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.651187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.655792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.658204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.700856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 834/1669] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:43 2022] POST /identity/v3/users => generated 304 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:43.701890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.715007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660011f3-5535-4602-b466-9184ecfc74fb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.720733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.721424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.721702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.721978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.722716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.724453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.740496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 835/1670] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:43.741783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.766989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303411c2-28b0-4f43-ac85-fc34639d8d37 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.779444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.780224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.780582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.780864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.781611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.783328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.811013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 836/1671] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:43.819457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:43.840983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400910f3-7e30-4260-97b9-5187283bd4f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.854939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4291fa-9983-43dd-b426-2272ee1f7554 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:43.857503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.857503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.857503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.857503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] PATH_INFO: `/v3/projects/f091fda3031345eb9cddde9eb600eb2d/users/8186b79485f0426a91de3de7fdd5ab59/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.878184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:43.879368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.879621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.879898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] PATH_INFO: `/v3/projects/402fa8d4c9224ad8a5028295e6a90483` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:43.885460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=402fa8d4c9224ad8a5028295e6a90483)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.891958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.918856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f091fda3031345eb9cddde9eb600eb2d, user_id=8186b79485f0426a91de3de7fdd5ab59, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:43.922766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21d65e-649d-434e-9299-855494cf5074 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:43.964689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 837/1672] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:43 2022] PUT /identity/v3/projects/f091fda3031345eb9cddde9eb600eb2d/users/8186b79485f0426a91de3de7fdd5ab59/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:43.973385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab9c7-4b6f-4def-8351-181d4cd574f5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:43.973702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab9c7-4b6f-4def-8351-181d4cd574f5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:43.974039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ab9c7-4b6f-4def-8351-181d4cd574f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.013904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c14ab9c7-4b6f-4def-8351-181d4cd574f5 None None] MFA Rules not processed for user `8186b79485f0426a91de3de7fdd5ab59`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.015334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c14ab9c7-4b6f-4def-8351-181d4cd574f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.030957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402fa8d4c9224ad8a5028295e6a90483', 'request_id': 'req-accbf3b8-3d0f-4891-b928-35293d88bfbb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:44.055544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402fa8d4c9224ad8a5028295e6a90483', 'request_id': 'req-accbf3b8-3d0f-4891-b928-35293d88bfbb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:44.061020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:44.073933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402fa8d4c9224ad8a5028295e6a90483', 'request_id': 'req-accbf3b8-3d0f-4891-b928-35293d88bfbb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:44.101580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] The token cache is being invalidate because project 402fa8d4c9224ad8a5028295e6a90483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:40:44.101880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:44.104727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-accbf3b8-3d0f-4891-b928-35293d88bfbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:44.108118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 836/1673] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:40:43 2022] DELETE /identity/v3/projects/402fa8d4c9224ad8a5028295e6a90483 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:40:44.113130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.113420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.113700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.125933 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:44.185130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] MFA Rules not processed for user `5ab5b939a13e43469b3a4a48da559c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.186380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.245672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 838/1674] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:44.247606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dbe33-d01c-49a3-b31e-6f3141221dc8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.247900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dbe33-d01c-49a3-b31e-6f3141221dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.248182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dbe33-d01c-49a3-b31e-6f3141221dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.276646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5d8dbe33-d01c-49a3-b31e-6f3141221dc8 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.283322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5d8dbe33-d01c-49a3-b31e-6f3141221dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.399831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 837/1675] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:44.400669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:44.408243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.447818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 839/1676] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:44.450265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954284c-9e98-4dae-b985-5caa0da9ca49 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.450557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954284c-9e98-4dae-b985-5caa0da9ca49 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.450823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954284c-9e98-4dae-b985-5caa0da9ca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.470151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49abc9-65d7-4c2d-b379-05884132de92 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:44.476043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:44.477154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.477653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.478871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.479918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:44.483443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:44.512423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6954284c-9e98-4dae-b985-5caa0da9ca49 None None] MFA Rules not processed for user `5ab5b939a13e43469b3a4a48da559c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.514180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6954284c-9e98-4dae-b985-5caa0da9ca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.526727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 840/1677] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:44.528842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.536637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 838/1678] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/projects => generated 407 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:44.537430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.537629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:44.538089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.587234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.591778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.694051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6584-3e8e-4283-84ef-deed836aef8a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:44.711806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:44.716157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.716462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.716747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.718401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:44.728282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96e9fc-9236-493d-87f6-a1086ce96221 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:44.770612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 841/1679] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:44.772214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:44.775287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.797545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 839/1680] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/users => generated 310 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:44.798929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.799254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.799540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.865078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] MFA Rules not processed for user `80113e82c4a74f4e8fd42b88aca3920d`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:44.866318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:44.912606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:44.926664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ade553-ca88-4358-a2b3-6adbe6ab83ee None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:44.942017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:44.942628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:44.943174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:44.943493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:44.956666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:45.034673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 840/1681] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:45.035550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.060817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.063957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.089464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 842/1682] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:45.090670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfadd01-595d-4ec6-a4db-9af96d358e39 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.093471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.103915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.104976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.105546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.106157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.107600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:45.108098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.111018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.117137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b387e49-9b28-48a3-8d69-513409bc6ccc service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.131520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.132215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.132504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.132789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.133535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.135450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.153807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 841/1683] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:45.155141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.157383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:45.167163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 843/1684] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:45.168143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.192822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc0a7d-d07e-410f-ab1b-d2b21d4c2dbd admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.206007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31821a-5265-477f-a41d-134c811ca02e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.206196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.208899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.209295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.209652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.212839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.213469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.213759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.214038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.214759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.217584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.223071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.225278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.242376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 842/1685] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:45.243456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.263719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358e836-10eb-4f9f-9b44-a18f9ce6f2f6 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.272328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.273672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.274090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.274420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] PATH_INFO: `/v3/projects/c8f7f192574a4993be2597c60279ead7/users/89a225342df9448a9e33734f5b54b26e/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.282363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 844/1686] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:45 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:45.286511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.314586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f7f192574a4993be2597c60279ead7, user_id=89a225342df9448a9e33734f5b54b26e, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.320455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.353018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 843/1687] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:45 2022] PUT /identity/v3/projects/c8f7f192574a4993be2597c60279ead7/users/89a225342df9448a9e33734f5b54b26e/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:45.356425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.484099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938b16-003c-4dfb-85b8-ec87709d9f8f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.508148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.509617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.511431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.511720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.512519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.522258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1022f-0810-446a-96be-72cd1d9afcb5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.522604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.542109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.542869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.543236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.543632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.560127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.563986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.591476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 845/1688] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:45 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:45.596098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.600804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 844/1689] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:45.605168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.615233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ac2f8-3e1a-4394-a119-2d32a882b147 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.624346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.625308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.626089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.626528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.627498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.630768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.639308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c9cd6-4809-4648-873f-e1fafbe00ca4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.648517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 846/1690] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:40:45.652401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.664536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.669973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.669973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.670170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] PATH_INFO: `/v3/projects/c8f7f192574a4993be2597c60279ead7/users/89a225342df9448a9e33734f5b54b26e/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.672801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f7f192574a4993be2597c60279ead7, user_id=89a225342df9448a9e33734f5b54b26e, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.673048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7f314-3ac9-471e-89b1-a3c6008cb3ee admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.675596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eae8a5-d26a-4855-a2de-15ee036f1c58 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.702425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.704568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 845/1691] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:45 2022] PUT /identity/v3/projects/c8f7f192574a4993be2597c60279ead7/users/89a225342df9448a9e33734f5b54b26e/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:45.710144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.710597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.711024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.712087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.713988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.717015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bbb1-f00b-4055-b99a-14f793244d91 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.720015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bbb1-f00b-4055-b99a-14f793244d91 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.720495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bbb1-f00b-4055-b99a-14f793244d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.734739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 847/1692] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:40:45.736497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:45.748757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0bd24-9beb-405d-a8c7-40957bee087e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:45.807333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7883bbb1-f00b-4055-b99a-14f793244d91 None None] MFA Rules not processed for user `89a225342df9448a9e33734f5b54b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:45.810877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7883bbb1-f00b-4055-b99a-14f793244d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:45.857295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:45.894299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:45.896149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.896562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.896914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] PATH_INFO: `/v3/projects/849f879480e4419db918647e4a310777/users/2e7d1d6eb4ad4b218be6dd3212e24e30/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.936659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849f879480e4419db918647e4a310777, user_id=2e7d1d6eb4ad4b218be6dd3212e24e30, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:45.939775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c3c7c-83e0-4f2f-b876-51df7bacbd77 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:45.986132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 848/1693] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:45 2022] PUT /identity/v3/projects/849f879480e4419db918647e4a310777/users/2e7d1d6eb4ad4b218be6dd3212e24e30/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:45.987855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778f16-2ec4-4463-8f97-a8a74d00463a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:45.988148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778f16-2ec4-4463-8f97-a8a74d00463a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:45.988429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778f16-2ec4-4463-8f97-a8a74d00463a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:45.998925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 849/1694] 10.222.0.74 () {54 vars in 912 bytes} [Thu Jun 16 16:40:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:46.002418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.002723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.003636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.018139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:46.055260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 846/1695] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:46.066435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.069386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.069716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.090242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] MFA Rules not processed for user `2e7d1d6eb4ad4b218be6dd3212e24e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:46.092192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:46.133483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:46.136382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:46.224892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 850/1696] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:46.224892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.224892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:46.229327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 847/1697] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:46.231996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.307598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029ab15-fd9a-452f-a5ee-c0308b8d418c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.316277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.316522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d27e-d135-4193-825e-c221251f52d8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.317229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.317613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.318064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.318878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.321109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.333951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.336681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.336975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.337258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.366165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 848/1698] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:46 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:46.367583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.369662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:46.462980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.472871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.478615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:46.491000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 851/1699] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:46 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:46.492107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.509422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9b39-f4cd-4488-8477-6151885120f2 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.569764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26cb9a-d62a-49cd-90c5-ab9eeb1c18c8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.581848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.583758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.585780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.586204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.599119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.600002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.600427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.600836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.601669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.604035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.614143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.615671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35093b6f-2f31-441c-b44e-788a63cac149 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.648788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 849/1700] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:46.650519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 852/1701] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:46 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:46.654966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.655362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.655992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.656146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.667927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bebb3-fc31-485a-a410-b4eb96ba3e31 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.685443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.686644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.687240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.687885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.689849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.690796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] MFA Rules not processed for user `b49313559188427bb9f42068583a8292`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:46.692517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:46.694764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.715465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 853/1702] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:40:46.720101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.737723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:46.741200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2057e-081c-475c-befe-8fe9c6e47f46 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.752791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.757059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.757880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.758458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.763677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.763677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.803783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 854/1703] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:46.808134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.825205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 850/1704] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:46.825487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e74e0-294c-4ec4-ae65-588e7c7c1437 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:46.838761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:46.840115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.843894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.845266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] PATH_INFO: `/v3/projects/40214004a6d0422390cb2019d424ad8b/users/f9de92d5b20f407d823cb4f67667a0c3/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:46.886595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40214004a6d0422390cb2019d424ad8b, user_id=f9de92d5b20f407d823cb4f67667a0c3, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:46.889976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503e7d2-8c1d-4350-a37b-104fc602eb68 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:46.920344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:46.928410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 855/1705] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:46 2022] PUT /identity/v3/projects/40214004a6d0422390cb2019d424ad8b/users/f9de92d5b20f407d823cb4f67667a0c3/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:40:46.940592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:46.941215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:46.941720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.004596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] MFA Rules not processed for user `f9de92d5b20f407d823cb4f67667a0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:47.008482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:47.014974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1453a9-d4e7-4aff-ac1f-be3dd3626943 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.131466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.132138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.132422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.132707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.166793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 856/1706] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:40:47.171757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.211976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.213986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.234434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:47.241514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 851/1707] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:47.276282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e890-0c0f-4199-9c3f-eba8826fadd3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.281768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.282685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.283335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.283900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.284749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.287375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.329696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 857/1708] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:47 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:40:47.333723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.347205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989284c-bcdb-4237-92f4-4655716d4a29 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.396999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:47.440695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.441637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.442378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.442800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.444489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.446906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.503791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 852/1709] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:47 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:47.505372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.525856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47839e9e-402b-45cf-8da8-3d5f75c79a59 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.542095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.542425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.542855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.543211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.545577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.551200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f5e81-49f8-49a3-8bb2-66b8b13ccd42 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.595413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 858/1710] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:47.599613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.615140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc22ebd-f1a9-47e7-8c97-08a42c4d852a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.624940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.626079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.626417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.626795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] PATH_INFO: `/v3/projects/c91a0711370545258632ffcb772ec318/users/6567ee0b1b934a55928f0f7f3043600a/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.672618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91a0711370545258632ffcb772ec318, user_id=6567ee0b1b934a55928f0f7f3043600a, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.674989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106919a-48e9-47a3-86f7-1d955d17b448 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.675379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106919a-48e9-47a3-86f7-1d955d17b448 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.675673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106919a-48e9-47a3-86f7-1d955d17b448 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.677150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 859/1711] 10.222.0.74 () {54 vars in 960 bytes} [Thu Jun 16 16:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:47.685003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.687606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.688185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.688773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.733206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 853/1712] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:47 2022] PUT /identity/v3/projects/c91a0711370545258632ffcb772ec318/users/6567ee0b1b934a55928f0f7f3043600a/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:47.734613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.756960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:47.762773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:47.905272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af032116-7e74-44b0-99de-e2df083895db admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.919440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.920897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.921190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.921476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:47.922232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:47.926127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:47.944579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 860/1713] 10.222.0.74 () {58 vars in 1057 bytes} [Thu Jun 16 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:47.946763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 854/1714] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:47.950983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.958916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:47.970453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7826d0c-a900-42b2-9e53-77c6c3e5181d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:47.990901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:47.992017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:47.992303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:47.992586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] PATH_INFO: `/v3/projects/c91a0711370545258632ffcb772ec318/users/6567ee0b1b934a55928f0f7f3043600a/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.003277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7e774-0e4a-4ded-914e-d585ede115ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:48.005766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91a0711370545258632ffcb772ec318, user_id=6567ee0b1b934a55928f0f7f3043600a, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:48.015803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950887f-3e23-4334-9cd8-6d0d548df04c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:48.029994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:48.030682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.031805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.031805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.037644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:48.064422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:48.064422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c2a81-ed04-439e-9733-c22eb0f9e8b9 service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:48.065532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 861/1715] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:47 2022] PUT /identity/v3/projects/c91a0711370545258632ffcb772ec318/users/6567ee0b1b934a55928f0f7f3043600a/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:48.067848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.068330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.068934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.090575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 855/1716] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:48.152608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] MFA Rules not processed for user `6567ee0b1b934a55928f0f7f3043600a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:48.154072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:48.281294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6037-d603-4a2b-8896-e68ae127fd3c None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.282065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6037-d603-4a2b-8896-e68ae127fd3c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.282598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e6037-d603-4a2b-8896-e68ae127fd3c None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.285024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 856/1717] 10.222.0.74 () {54 vars in 912 bytes} [Thu Jun 16 16:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:40:48.314308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.318674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.319207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.345080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 862/1718] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:48.356360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:48.382507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:48.388188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:48.406640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:48.490882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c7973-b672-4131-880a-fe0a948dc9d7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:48.497884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:48.498658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.499005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.499392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.525257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:48.527193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:48.562658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 863/1719] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:48.630114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:48.632042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:48.644582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 857/1720] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:40:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:48.684345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d327a9-b2a1-4b25-b3fc-6a1c6a0cbea1 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:48.697070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:48.698346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:48.705147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:48.716017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:48.778546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:48.787337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:48.812049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 864/1721] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:40:49.915092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:49.936746 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a2c6a-1def-4c2e-9526-1831882b775b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:49.968695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:49.969388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:49.969677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:49.969949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:50.099768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:50.198664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:50.200313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d70039-f361-4c43-83f6-a56454b4cf92 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:50.232784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 858/1722] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:50.916203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:50.918528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:51.287608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ac865-ee7e-403c-8964-5ce7bb9d9a3e service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:51.310439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:51.319168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:51.319168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:51.319168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] PATH_INFO: `/v3/users/d9d03b65829f4e1d8e497c6c30ccfcdf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:51.321547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d03b65829f4e1d8e497c6c30ccfcdf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:51.324888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:51.425023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24cfde-2db4-4489-b713-546c623e6f31 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:51.425641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24cfde-2db4-4489-b713-546c623e6f31 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:51.426074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24cfde-2db4-4489-b713-546c623e6f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:51.464469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9e24cfde-2db4-4489-b713-546c623e6f31 None None] MFA Rules not processed for user `80113e82c4a74f4e8fd42b88aca3920d`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:51.467385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9e24cfde-2db4-4489-b713-546c623e6f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:51.568516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d03b65829f4e1d8e497c6c30ccfcdf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:51.570698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d03b65829f4e1d8e497c6c30ccfcdf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:51.601362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d03b65829f4e1d8e497c6c30ccfcdf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:51.653713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d03b65829f4e1d8e497c6c30ccfcdf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:51.672596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7aeccb4b-f84c-4549-8edf-855b08f3102d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d03b65829f4e1d8e497c6c30ccfcdf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:51.686107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 865/1723] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:40:50 2022] DELETE /identity/v3/users/d9d03b65829f4e1d8e497c6c30ccfcdf => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:51.690618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:51.691752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:51.692137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:51.717328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 859/1724] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:51.774035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:51.787494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:51.854302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:51.877399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:51.878183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:51.878604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:52.003724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:52.018739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:52.024427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 866/1725] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:52.035874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:52.039246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:52.191306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db2b72-5024-419d-bf61-eba052f75332 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:52.198593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 860/1726] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:40:52.212736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:52.213406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:52.213695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:52.213979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:52.226057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:52.227707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:52.239921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 867/1727] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:40:52.383877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:52.385877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:52.661292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880ce8-0099-4281-906d-5ee4cea78a55 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:52.703457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:52.704162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:52.707519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:52.707863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:52.730430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:52.732776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ede26-31f4-4b6a-bb7e-55298a57915a service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:52.744320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 861/1728] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:40:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:40:52.825646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:52.828693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:52.869092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17925658-f69a-428b-972b-2c74f5476ac2 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:52.880948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:52.881753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:52.882102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:52.882317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] PATH_INFO: `/v3/projects/e136d4cbac6e44a59aebfc722a03b7e6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:52.883897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e136d4cbac6e44a59aebfc722a03b7e6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:52.885774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.003091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e136d4cbac6e44a59aebfc722a03b7e6', 'request_id': 'req-c01a413f-a5e7-4705-bb51-4e29ede99be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:53.023694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e136d4cbac6e44a59aebfc722a03b7e6', 'request_id': 'req-c01a413f-a5e7-4705-bb51-4e29ede99be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:53.041348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e136d4cbac6e44a59aebfc722a03b7e6', 'request_id': 'req-c01a413f-a5e7-4705-bb51-4e29ede99be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:53.064468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] The token cache is being invalidate because project e136d4cbac6e44a59aebfc722a03b7e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:40:53.065130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:53.066168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c01a413f-a5e7-4705-bb51-4e29ede99be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:40:53.069395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 868/1729] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:40:52 2022] DELETE /identity/v3/projects/e136d4cbac6e44a59aebfc722a03b7e6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:40:53.074387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.074779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.075537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.104592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:53.106007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:53.118655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:53.237208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 862/1730] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:53.247830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.248424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.249672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.275249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:53.276617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:53.287024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 869/1731] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:53.291177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.303128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242aef2d-53bc-4466-8d6a-d69c8380ddde None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.308708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.309639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.310028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.310539 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.311593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:40:53.312548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.314720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.331906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 863/1732] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:40:53.335527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.347337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb217d-4c6d-4699-a415-f39c38481514 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.354398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.355439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.355992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.356491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.357514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.359947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.405574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 870/1733] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:40:53 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:40:53.409265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.425404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9a7d-1808-4a6b-b4c1-a9e60bb3f298 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.501749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.510783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.511270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.511698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.513249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.518580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.558207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 864/1734] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:40:53 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:40:53.559772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.574961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a13d12-84d7-49f4-9e72-014e4c46a78e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.579885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.580582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.580864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.581300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.582171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.583566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.601256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 871/1735] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:40:53.605205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.615807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde57206-9b00-4610-afcd-35d692d01d88 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.621957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.622651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.622935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.623246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.623987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.625641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca819b4-ff37-4dfe-b1bc-fc1758033b1c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.637391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 865/1736] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:40:53.641697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.652395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e4808-fd52-4ba6-9a90-04e7d498bbc0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:53.657357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:53.658259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.658542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.658815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] PATH_INFO: `/v3/projects/2c460e2af02e4a98830f7240d4e4d4f3/users/18f859d8d5884bab99a556f00d323f6d/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.667939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:40:53.680827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c460e2af02e4a98830f7240d4e4d4f3, user_id=18f859d8d5884bab99a556f00d323f6d, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:53.684012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:53.703251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 872/1737] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:40:53 2022] PUT /identity/v3/projects/2c460e2af02e4a98830f7240d4e4d4f3/users/18f859d8d5884bab99a556f00d323f6d/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:40:53.712348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:53.712834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:53.712906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:53.751719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] MFA Rules not processed for user `18f859d8d5884bab99a556f00d323f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:40:53.753121 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:53.864966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 866/1738] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:40:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:40:53.877338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:40:53.879869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:40:54.010095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd286b1-898c-41a2-b43d-d96a262df8af admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:40:54.015479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:40:54.016180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:40:54.016545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:40:54.016810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:40:54.026481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:40:54.028350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0126c0-bfaf-4ac7-893a-e34fa3e3533d service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:40:54.040121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 873/1739] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:40:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:00.095543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:00.097013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:00.118321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:00.229039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f34ec8-c33a-4f84-a4ec-e7d34bb0386e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:00.235001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:00.236143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:00.236474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:00.236883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] PATH_INFO: `/v3/users/556cbb2920b448dc81bc90606cfc59f4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:00.238098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556cbb2920b448dc81bc90606cfc59f4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:00.240422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:00.338427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556cbb2920b448dc81bc90606cfc59f4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:00.339781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556cbb2920b448dc81bc90606cfc59f4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:00.355944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556cbb2920b448dc81bc90606cfc59f4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:00.375987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556cbb2920b448dc81bc90606cfc59f4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:00.411591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556cbb2920b448dc81bc90606cfc59f4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:00.430513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 867/1740] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:00 2022] DELETE /identity/v3/users/556cbb2920b448dc81bc90606cfc59f4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:00.435717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:00.436776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:00.437251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:00.466070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:00.467550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:00.591118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 874/1741] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:00.604721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:00.607188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:00.718128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa77d4-23b0-4970-8cc2-17bda192f603 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:00.722412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:00.723116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:00.723468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:00.723749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:00.734759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:00.736206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dfa69-fec8-43c3-8601-37a77b510f18 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:00.746873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 868/1742] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:01.181282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.183274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:01.210091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505d5e-1bc0-48b9-b3dd-97bacc1f2140 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.220673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.222403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.222765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.223264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] PATH_INFO: `/v3/projects/28061579532249b7922ab14c12529580` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.224701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28061579532249b7922ab14c12529580)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.227136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.320159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28061579532249b7922ab14c12529580', 'request_id': 'req-533d286c-db42-4593-8956-9bad53fab405'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:01.330605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28061579532249b7922ab14c12529580', 'request_id': 'req-533d286c-db42-4593-8956-9bad53fab405'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:01.339489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28061579532249b7922ab14c12529580', 'request_id': 'req-533d286c-db42-4593-8956-9bad53fab405'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:01.361651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] The token cache is being invalidate because project 28061579532249b7922ab14c12529580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:01.362432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:01.362748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-533d286c-db42-4593-8956-9bad53fab405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:01.368809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 875/1743] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:01 2022] DELETE /identity/v3/projects/28061579532249b7922ab14c12529580 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:01.375128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.375797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.375914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.426482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:01.427944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:01.478883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:01.518971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 869/1744] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:01.529077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.529637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.529728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.551447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:01.552694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:01.561282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 876/1745] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:01.565611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.577144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cbe5-4002-4b16-8f85-825d5b7de45a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.582489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.583252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.583672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.583960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.584738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:01.585051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.586769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.604113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 870/1746] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:01.608418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.621928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02888-9d53-4840-8b12-e1b47587f60f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.627313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.628097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.628457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.628809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.629910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.631962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.636359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:01.673127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 877/1747] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:01 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:01.674680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.697160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eaff8-672a-4071-b05a-3f34c25f4d25 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.802228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.802941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.803948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.803948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.804638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.815572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.862450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 871/1748] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:01 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:01.866974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.889502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8709ec-d6c8-4220-b730-f2805bdde0bb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.894192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.894918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.895310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.895506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.896267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.897918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.911093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 878/1749] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:01.914203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.924625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6ad97-81c9-4a2f-8624-6796c65d166c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.930232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.931032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.931344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.931714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.932583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:01.934474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c3ef-f088-4c05-b32d-842b8cc70622 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:01.945463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 872/1750] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:01.949689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:01.964680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba9d3d-ea06-41ba-8454-93c716f2b84a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:01.971423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:01.972519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:01.972904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:01.973292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] PATH_INFO: `/v3/projects/55531cdf1c1a4dc5be637839b3e6d464/users/e4c42e4b0eeb421e91c61fe2b4af6a59/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:01.998057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55531cdf1c1a4dc5be637839b3e6d464, user_id=e4c42e4b0eeb421e91c61fe2b4af6a59, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.000936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.019971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 879/1751] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:01 2022] PUT /identity/v3/projects/55531cdf1c1a4dc5be637839b3e6d464/users/e4c42e4b0eeb421e91c61fe2b4af6a59/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:02.030308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.030590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.030873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.063586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] MFA Rules not processed for user `e4c42e4b0eeb421e91c61fe2b4af6a59`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:02.064800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:02.142840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 873/1752] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:02.143760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.255821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba117f-ef26-49d6-9984-29abd2ed3913 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.261599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.262319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.262606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.262884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.263637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.265624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.299569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 880/1753] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:02 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:02.300673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.319283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba8d40-cb63-476b-924f-262a48b34e23 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.343706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:02.398939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.399874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.400034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.400341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.401238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.403568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.425729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 874/1754] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:02 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:02.429868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.441525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14aa7f-38b9-490c-bd47-dbafd0d44a5c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.448146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.448919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.449271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.449593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.450649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.452326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.464055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 881/1755] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:02.468384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.480508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0742682-3314-496f-bb9d-fa572ff937ed admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.487014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.487338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.487649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.487936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] PATH_INFO: `/v3/projects/b11c89848be643ed929311e811795afb/users/20f52f9811fb4ffa9f1571a538a94582/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.506080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11c89848be643ed929311e811795afb, user_id=20f52f9811fb4ffa9f1571a538a94582, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.508822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.525095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 875/1756] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:02 2022] PUT /identity/v3/projects/b11c89848be643ed929311e811795afb/users/20f52f9811fb4ffa9f1571a538a94582/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:02.530160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.545775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120b1c6-be99-4047-b9e1-4273c5114fb4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.639554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.640253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.640555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.640916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.641570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.643311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cd864-95a8-4c4d-859d-504d144d6c61 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.660582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 882/1757] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:02.662035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.675087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda358de-dd7a-4974-9517-979b74fc334b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.686995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.687943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.688260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.688535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] PATH_INFO: `/v3/projects/b11c89848be643ed929311e811795afb/users/20f52f9811fb4ffa9f1571a538a94582/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.690597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11c89848be643ed929311e811795afb, user_id=20f52f9811fb4ffa9f1571a538a94582, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:02.693713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:02.713845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 876/1758] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:02 2022] PUT /identity/v3/projects/b11c89848be643ed929311e811795afb/users/20f52f9811fb4ffa9f1571a538a94582/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:02.725046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.725046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.725046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:02.753991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] MFA Rules not processed for user `20f52f9811fb4ffa9f1571a538a94582`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:02.755571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:02.764511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:02.853481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 883/1759] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:02.866666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:02.869116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:02.974778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a9239-a08d-433f-a174-7174b8c50cea admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:02.980088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:02.981122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:02.982022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:02.982311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:03.001714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:03.006589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:03.024537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 877/1760] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:03.178085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:03.193997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1f92a-fe9e-4408-839d-795d636d516b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:03.210632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:03.211361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:03.211712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:03.212079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:03.303289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:03.304756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:03.315080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 884/1761] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:03.762590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:03.764912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:03.777489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb74182-2072-442d-b69d-080810046a98 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:03.781551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:03.782181 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:03.782462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:03.782735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:03.784870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:03.847860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:03.849711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aead0d-4e91-4040-b722-4d2d1ef3e134 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:03.860299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 878/1762] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:04.549190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:04.550777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:04.667513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcd45d-d1b3-4dd3-8701-e76bccfb5794 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:04.678197 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:04.679385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:04.681693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:04.682165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] PATH_INFO: `/v3/users/8186b79485f0426a91de3de7fdd5ab59` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:04.684790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8186b79485f0426a91de3de7fdd5ab59)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:04.687641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:04.840325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8186b79485f0426a91de3de7fdd5ab59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:04.842386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:04.844085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8186b79485f0426a91de3de7fdd5ab59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:04.873699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8186b79485f0426a91de3de7fdd5ab59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:04.922515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8186b79485f0426a91de3de7fdd5ab59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:04.933507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8186b79485f0426a91de3de7fdd5ab59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:04.944505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 885/1763] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:04 2022] DELETE /identity/v3/users/8186b79485f0426a91de3de7fdd5ab59 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:04.954362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:04.954362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:04.954362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:04.987044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:04.989552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:05.008255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:05.115398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 879/1764] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:05.125403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:05.126954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:05.245854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c20f-8817-4354-a8b7-119f81f8a163 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:05.264180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:05.264826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:05.265301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:05.265577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:05.284829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:05.287190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:05.303546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 886/1765] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:05.665222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:05.666710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:05.679486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c03b-ec66-4a2f-801a-93e824de5b91 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:05.684462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:05.685910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:05.686287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:05.686751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] PATH_INFO: `/v3/projects/f091fda3031345eb9cddde9eb600eb2d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:05.688919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f091fda3031345eb9cddde9eb600eb2d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:05.691005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:05.760976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f091fda3031345eb9cddde9eb600eb2d', 'request_id': 'req-bb395d89-6d95-457c-9707-fc14dfc6bfdb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.767152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f091fda3031345eb9cddde9eb600eb2d', 'request_id': 'req-bb395d89-6d95-457c-9707-fc14dfc6bfdb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.776501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f091fda3031345eb9cddde9eb600eb2d', 'request_id': 'req-bb395d89-6d95-457c-9707-fc14dfc6bfdb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.794718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] The token cache is being invalidate because project f091fda3031345eb9cddde9eb600eb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:05.795105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.795807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.797758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 880/1766] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:05 2022] DELETE /identity/v3/projects/f091fda3031345eb9cddde9eb600eb2d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:05.801824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:05.813453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f095a6-bb61-4d51-b7b2-515569533631 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:05.901854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:05.902887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:05.903893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:05.904418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] PATH_INFO: `/v3/users/89a225342df9448a9e33734f5b54b26e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:05.906005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a225342df9448a9e33734f5b54b26e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:05.908224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:05.975974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a225342df9448a9e33734f5b54b26e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.976674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a225342df9448a9e33734f5b54b26e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.987139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a225342df9448a9e33734f5b54b26e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:05.999409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:06.009146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a225342df9448a9e33734f5b54b26e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.022609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5104d005-add1-42a5-b17b-c1b62ff20721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a225342df9448a9e33734f5b54b26e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.034535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 887/1767] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:05 2022] DELETE /identity/v3/users/89a225342df9448a9e33734f5b54b26e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:06.409046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:06.410336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.427409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb395d89-6d95-457c-9707-fc14dfc6bfdb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:06.495406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:06.496462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.496717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.497036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] PATH_INFO: `/v3/projects/c8f7f192574a4993be2597c60279ead7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.498325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f7f192574a4993be2597c60279ead7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:06.500499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:06.561508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:06.567144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f7f192574a4993be2597c60279ead7', 'request_id': 'req-32f04aad-23e7-4088-99a9-1d71a12b74ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.579652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f7f192574a4993be2597c60279ead7', 'request_id': 'req-32f04aad-23e7-4088-99a9-1d71a12b74ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.590847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f7f192574a4993be2597c60279ead7', 'request_id': 'req-32f04aad-23e7-4088-99a9-1d71a12b74ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.607439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] The token cache is being invalidate because project c8f7f192574a4993be2597c60279ead7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:06.607714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.611868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.613642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 881/1768] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:06 2022] DELETE /identity/v3/projects/c8f7f192574a4993be2597c60279ead7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:06.618463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d25ef-6650-47fd-b697-cc083373c438 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.618788 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d25ef-6650-47fd-b697-cc083373c438 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.619104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d25ef-6650-47fd-b697-cc083373c438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.638796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f28d25ef-6650-47fd-b697-cc083373c438 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:06.640079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f28d25ef-6650-47fd-b697-cc083373c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.658563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:06.670330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.712324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 888/1769] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:06.720274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.720581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.720865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.740757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:06.742165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.750814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 889/1770] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:06.755315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:06.756843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.769170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892279af-6a98-4555-b5dc-7bd9dc9fde3f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:06.774286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:06.775233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.775545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.775809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] PATH_INFO: `/v3/users/e4c42e4b0eeb421e91c61fe2b4af6a59` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.777103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c42e4b0eeb421e91c61fe2b4af6a59)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:06.779176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:06.832105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c42e4b0eeb421e91c61fe2b4af6a59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.832839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c42e4b0eeb421e91c61fe2b4af6a59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.840380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c42e4b0eeb421e91c61fe2b4af6a59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.854841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c42e4b0eeb421e91c61fe2b4af6a59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.863039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c42e4b0eeb421e91c61fe2b4af6a59'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.867247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f04aad-23e7-4088-99a9-1d71a12b74ec admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:06.869248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 890/1771] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:06 2022] DELETE /identity/v3/users/e4c42e4b0eeb421e91c61fe2b4af6a59 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:06.872497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:06.872497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.873441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:06.874340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.874629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.874908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] PATH_INFO: `/v3/users/2e7d1d6eb4ad4b218be6dd3212e24e30` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.876163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7d1d6eb4ad4b218be6dd3212e24e30)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:06.878115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:06.926009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7d1d6eb4ad4b218be6dd3212e24e30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.926649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7d1d6eb4ad4b218be6dd3212e24e30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.927550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:06.933982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7d1d6eb4ad4b218be6dd3212e24e30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.941915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7d1d6eb4ad4b218be6dd3212e24e30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.944018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff63c6-2116-4377-8960-046835dc07cc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:06.949179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:06.950339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.950339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.950644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.951413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:06.951733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:06.953370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5787f2-fe7f-4966-aa0c-3b4be121c8e2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:06.955025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ebada8ed-d70c-47bb-9961-d2de0cb4bb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7d1d6eb4ad4b218be6dd3212e24e30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:06.961241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 882/1772] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:06 2022] DELETE /identity/v3/users/2e7d1d6eb4ad4b218be6dd3212e24e30 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:06.962518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.962810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.963117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.971997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 891/1773] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:06.971997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:06.971997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:06.971997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:06.982356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:06.983642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:06.989400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:06.990617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.032691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:07.049716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 883/1774] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:07.053783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.054981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.057830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 892/1775] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:07.061601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.062910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.067882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc0703-2221-44bf-88f6-ec4449c0e60a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.072472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.073136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.073418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.074087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.074860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.077185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.105383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 884/1776] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:07 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:07.106160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.108115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.126591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbdc63-f272-4ba1-ad37-a13a6ace7d04 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.181780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce9c7f-b21c-44a2-8d2b-3c6eef622980 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.182861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.183773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.184453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.184922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.185704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.186589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.186877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.187190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.286844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:07.296372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.298311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.302140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.303764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.312906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 885/1777] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:07.313579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.316547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 893/1778] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:07.324853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4024d-4c76-42de-abe5-e787c577d271 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.330416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.331141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.331432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.331712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.332421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.334922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.364213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 886/1779] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:07 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:07.368704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.381794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc7b15-2bfa-4550-850f-736ff67344c0 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.388833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.389585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.389957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.390262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.391991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.393387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.407038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 894/1780] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:07.411596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.425424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c74358-d85b-4b43-8fbf-6bffe357a310 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.437583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.439170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.440761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.441734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.444815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.450727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbad8d0-1199-47d3-b937-e02d6379ef19 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.473357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 887/1781] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:07.477327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.506826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0060d0-55ed-45d3-b767-e6982934e8cc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.514117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.515297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.515768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.516055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] PATH_INFO: `/v3/projects/34f24b129fc64b72897520a13e43d21e/users/29285267cf0d4f7cb0abdacabecb04d2/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.546692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f24b129fc64b72897520a13e43d21e, user_id=29285267cf0d4f7cb0abdacabecb04d2, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.553938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.597671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 895/1782] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:07 2022] PUT /identity/v3/projects/34f24b129fc64b72897520a13e43d21e/users/29285267cf0d4f7cb0abdacabecb04d2/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:07.609863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.610715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.610715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.647236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] MFA Rules not processed for user `29285267cf0d4f7cb0abdacabecb04d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:07.667125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.761460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:07.822106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 888/1783] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:07.836839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.841559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.849979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.861709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.935546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625a8f4-d36f-4a28-9f24-56a42dec80cd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.940070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.941425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.941873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.942284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.949452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93edf8-3096-4c21-97c1-563b5d66f340 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.955118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.955945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.956573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.956859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.957143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.957447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] PATH_INFO: `/v3/projects/849f879480e4419db918647e4a310777` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.958659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=849f879480e4419db918647e4a310777)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.960947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:07.966108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 896/1784] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:07.966814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:07.968648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:07.982779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d71c89-a789-44ef-bb69-9e5c54177821 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:07.987398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:07.988277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:07.988555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:07.988824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] PATH_INFO: `/v3/projects/55531cdf1c1a4dc5be637839b3e6d464` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:07.990002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55531cdf1c1a4dc5be637839b3e6d464)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:07.991967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.014498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849f879480e4419db918647e4a310777', 'request_id': 'req-b8d85169-a93b-435c-a600-bb0731462cea'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.020111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849f879480e4419db918647e4a310777', 'request_id': 'req-b8d85169-a93b-435c-a600-bb0731462cea'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.028156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849f879480e4419db918647e4a310777', 'request_id': 'req-b8d85169-a93b-435c-a600-bb0731462cea'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.039682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55531cdf1c1a4dc5be637839b3e6d464', 'request_id': 'req-244ee3d0-253b-4cec-b38c-ec0acbaca456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.040107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] The token cache is being invalidate because project 849f879480e4419db918647e4a310777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:08.040398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.041132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.042920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 889/1785] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:07 2022] DELETE /identity/v3/projects/849f879480e4419db918647e4a310777 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.044207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.045902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.048300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55531cdf1c1a4dc5be637839b3e6d464', 'request_id': 'req-244ee3d0-253b-4cec-b38c-ec0acbaca456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.054819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55531cdf1c1a4dc5be637839b3e6d464', 'request_id': 'req-244ee3d0-253b-4cec-b38c-ec0acbaca456'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.065842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] The token cache is being invalidate because project 55531cdf1c1a4dc5be637839b3e6d464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:08.066792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.067505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.071964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 897/1786] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:07 2022] DELETE /identity/v3/projects/55531cdf1c1a4dc5be637839b3e6d464 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.071964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.071964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.114922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85169-a93b-435c-a600-bb0731462cea admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.118417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:08.128566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:08.140887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ee3d0-253b-4cec-b38c-ec0acbaca456 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.145669 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:08.146596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.146884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.147200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] PATH_INFO: `/v3/users/f9de92d5b20f407d823cb4f67667a0c3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.148438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9de92d5b20f407d823cb4f67667a0c3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:08.150402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.179428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:08.180446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.181198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.181198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] PATH_INFO: `/v3/users/29285267cf0d4f7cb0abdacabecb04d2` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.182316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29285267cf0d4f7cb0abdacabecb04d2)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:08.184753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.199711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9de92d5b20f407d823cb4f67667a0c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.200612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9de92d5b20f407d823cb4f67667a0c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.207806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9de92d5b20f407d823cb4f67667a0c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.222075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9de92d5b20f407d823cb4f67667a0c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.230255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9de92d5b20f407d823cb4f67667a0c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.235791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29285267cf0d4f7cb0abdacabecb04d2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.236459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29285267cf0d4f7cb0abdacabecb04d2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.237470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 898/1787] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/users/f9de92d5b20f407d823cb4f67667a0c3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.241671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.242874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.249031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29285267cf0d4f7cb0abdacabecb04d2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.258227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29285267cf0d4f7cb0abdacabecb04d2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.271799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8d08ba1c-7c5c-4685-8ac4-32030c525f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29285267cf0d4f7cb0abdacabecb04d2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.283344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 890/1788] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/users/29285267cf0d4f7cb0abdacabecb04d2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.288255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.288547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.288829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.307892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:08.309984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.328223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d1d51-a47c-4b28-8238-c62078f0631d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.380790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 891/1789] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:08.392193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.393859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.405300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:08.406383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.406790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.407216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] PATH_INFO: `/v3/users/20f52f9811fb4ffa9f1571a538a94582` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.408943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f52f9811fb4ffa9f1571a538a94582)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:08.412369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.428774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:08.444641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:08.468160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1315-8dae-465d-b11f-44bb31c04461 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.472278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:08.472945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.473235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.473514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.483391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:08.484851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.485579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f52f9811fb4ffa9f1571a538a94582'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.487067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f52f9811fb4ffa9f1571a538a94582'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.496894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f52f9811fb4ffa9f1571a538a94582'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.498396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 892/1790] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:08.510209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f52f9811fb4ffa9f1571a538a94582'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.522624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f52f9811fb4ffa9f1571a538a94582'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.531301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 899/1791] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/users/20f52f9811fb4ffa9f1571a538a94582 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.617936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.619806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.642270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbcc74-20e0-40c0-a7b6-d14f51d8cf34 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.756235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:08.757402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:08.757826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:08.758192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] PATH_INFO: `/v3/projects/40214004a6d0422390cb2019d424ad8b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:08.759739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=40214004a6d0422390cb2019d424ad8b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:08.762001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:08.841589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40214004a6d0422390cb2019d424ad8b', 'request_id': 'req-e714629c-14f1-4867-b594-99755808d91b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.853086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40214004a6d0422390cb2019d424ad8b', 'request_id': 'req-e714629c-14f1-4867-b594-99755808d91b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.863114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40214004a6d0422390cb2019d424ad8b', 'request_id': 'req-e714629c-14f1-4867-b594-99755808d91b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.900260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] The token cache is being invalidate because project 40214004a6d0422390cb2019d424ad8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:08.900560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.904838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:08.907463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 893/1792] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/projects/40214004a6d0422390cb2019d424ad8b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:08.912053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.914560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:08.933547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb215101-f84a-4fde-ab60-be5e43eebfe8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:08.978388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:08.979857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.153282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714629c-14f1-4867-b594-99755808d91b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:09.176024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.178476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.178838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.179228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] PATH_INFO: `/v3/projects/b11c89848be643ed929311e811795afb` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.183449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11c89848be643ed929311e811795afb)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.185491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.195308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.196212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.196499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.196780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] PATH_INFO: `/v3/users/6567ee0b1b934a55928f0f7f3043600a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.199538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6567ee0b1b934a55928f0f7f3043600a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.201938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.216313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:09.281624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11c89848be643ed929311e811795afb', 'request_id': 'req-6434aaa5-6147-4d91-8f51-a2502d02dff6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.286269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6567ee0b1b934a55928f0f7f3043600a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.287666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6567ee0b1b934a55928f0f7f3043600a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.290941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11c89848be643ed929311e811795afb', 'request_id': 'req-6434aaa5-6147-4d91-8f51-a2502d02dff6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.296806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6567ee0b1b934a55928f0f7f3043600a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.302079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11c89848be643ed929311e811795afb', 'request_id': 'req-6434aaa5-6147-4d91-8f51-a2502d02dff6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.314461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6567ee0b1b934a55928f0f7f3043600a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.320012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] The token cache is being invalidate because project b11c89848be643ed929311e811795afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:09.320620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.321446 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.323686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 894/1793] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/projects/b11c89848be643ed929311e811795afb => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:09.324476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:09.324902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ab3e2a4b-f369-427e-8a6f-6ca8b84fa27d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6567ee0b1b934a55928f0f7f3043600a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.326184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.335718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 900/1794] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:08 2022] DELETE /identity/v3/users/6567ee0b1b934a55928f0f7f3043600a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:09.337791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8ae1-beb9-4307-a468-3e79d31702f5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.338596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8ae1-beb9-4307-a468-3e79d31702f5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.339922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e8ae1-beb9-4307-a468-3e79d31702f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.357649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:09.365369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-856e8ae1-beb9-4307-a468-3e79d31702f5 None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:09.368218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-856e8ae1-beb9-4307-a468-3e79d31702f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.490497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 901/1795] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 16 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:09.491769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.491913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434aaa5-6147-4d91-8f51-a2502d02dff6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:09.492382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.492461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.522209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:09.523776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.606089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 902/1796] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:09.608380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:09.610687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.639247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2ea42-b4d3-41ad-b0f0-950e2f78190d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:09.642161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.644422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.645159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.645809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] PATH_INFO: `/v3/projects/34f24b129fc64b72897520a13e43d21e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.649715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f24b129fc64b72897520a13e43d21e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.650353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.652902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.653413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.654398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.655750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.675247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.678620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73e194-5ea7-4bdb-96be-217fbd10477b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.691763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 903/1797] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:09.698095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63961f70-3e60-4784-bef9-3e37ead17c76 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.703756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63961f70-3e60-4784-bef9-3e37ead17c76 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.704162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63961f70-3e60-4784-bef9-3e37ead17c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.736145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-63961f70-3e60-4784-bef9-3e37ead17c76 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:09.738008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-63961f70-3e60-4784-bef9-3e37ead17c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.746173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f24b129fc64b72897520a13e43d21e', 'request_id': 'req-3010bd21-027e-4788-a9c1-e2357b2c8c9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.747639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:09.755613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f24b129fc64b72897520a13e43d21e', 'request_id': 'req-3010bd21-027e-4788-a9c1-e2357b2c8c9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.766173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f24b129fc64b72897520a13e43d21e', 'request_id': 'req-3010bd21-027e-4788-a9c1-e2357b2c8c9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.782868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] The token cache is being invalidate because project 34f24b129fc64b72897520a13e43d21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:09.783228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.783978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:09.785891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 895/1798] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:09 2022] DELETE /identity/v3/projects/34f24b129fc64b72897520a13e43d21e => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:09.786491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:09.787972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.842709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 904/1799] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:09.842709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.842709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.842709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.871063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:09.873098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.888095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 905/1800] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:09.889051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:09.909578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010bd21-027e-4788-a9c1-e2357b2c8c9c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:09.925836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f292467-c4f6-440d-8b43-1f8e1421e81b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:09.927783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.929010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.930249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.930537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] PATH_INFO: `/v3/users/18f859d8d5884bab99a556f00d323f6d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.935476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f859d8d5884bab99a556f00d323f6d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.947395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:09.947758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:09.949944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.950832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:09.951148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:09.953625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:09.953951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:09.957554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:09.971548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 906/1801] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:09.972479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:09.976420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:09.994093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6328b-8431-45b6-9b54-91e10840716d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.001267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.002237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.002529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.002826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] PATH_INFO: `/v3/projects/c91a0711370545258632ffcb772ec318` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.004379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91a0711370545258632ffcb772ec318)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.006889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.036335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:10.045800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f859d8d5884bab99a556f00d323f6d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.047813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f859d8d5884bab99a556f00d323f6d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.061997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f859d8d5884bab99a556f00d323f6d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.076895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f859d8d5884bab99a556f00d323f6d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.079836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91a0711370545258632ffcb772ec318', 'request_id': 'req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.097655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91a0711370545258632ffcb772ec318', 'request_id': 'req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.115042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91a0711370545258632ffcb772ec318', 'request_id': 'req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.115263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f859d8d5884bab99a556f00d323f6d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.132222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] The token cache is being invalidate because project c91a0711370545258632ffcb772ec318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:10.132453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.133072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5b09f000-eb25-4e8a-b037-5d440ef2e1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:10.135573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 907/1802] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:09 2022] DELETE /identity/v3/projects/c91a0711370545258632ffcb772ec318 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:10.137342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfe02-fafd-4504-95cf-12d842d1fe30 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.137744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfe02-fafd-4504-95cf-12d842d1fe30 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.138216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfe02-fafd-4504-95cf-12d842d1fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.143690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 896/1803] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:09 2022] DELETE /identity/v3/users/18f859d8d5884bab99a556f00d323f6d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:10.146936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.151347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.162917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bffcfe02-fafd-4504-95cf-12d842d1fe30 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:10.163400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bffcfe02-fafd-4504-95cf-12d842d1fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.249119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:10.256979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e4e5-14f3-48be-bab3-46235a167a84 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.265776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 908/1804] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:10.268446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.268701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.269068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.270427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.271514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.272777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.273199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.273996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.284576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b080-0492-493d-b2a9-3e437e93fe77 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.324033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:10.329226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.350908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 897/1805] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:10.354486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d45e-5894-4ae8-bc8b-5511aff92517 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.355083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d45e-5894-4ae8-bc8b-5511aff92517 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.355680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d45e-5894-4ae8-bc8b-5511aff92517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.359613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 909/1806] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:10.360497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.383103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63130a-a4f1-4392-8a22-13ef51c795ea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.396930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-41b6d45e-5894-4ae8-bc8b-5511aff92517 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:10.399589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-41b6d45e-5894-4ae8-bc8b-5511aff92517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.474740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.475587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.475930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.476221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.477178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:10.477495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.479166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.495823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 910/1807] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:10.497344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.499812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.520591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 898/1808] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:10.522773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.523284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.523769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.523871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536849c1-1257-468d-a46f-782966ce7a4e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.532509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.533278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.533569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.533908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.535545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.540414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.557456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:10.560039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.569850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 899/1809] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:10.570785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.572744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 911/1810] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/users => generated 292 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:10.573914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.577454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:10.586665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78224abe-c3bb-435e-9755-10d492d0b9b2 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.592514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.593418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.593797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.594172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.594967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.597779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.613629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:10.682164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 900/1811] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:10.683547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013538ec-938f-4530-ad9f-03d7e4b9f61c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.687279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.779699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.780740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.781209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.781663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.851010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558652a-7f74-40c0-9f46-cdab50565288 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.868047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.868753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.869044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.869326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.870075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:10.870404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.872638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.923242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 901/1812] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:10.924588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:10.953774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f784d1-fc7b-44ae-a8c4-da84a55ad294 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:10.977346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:10.977563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:10.977686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:10.978007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:10.979091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:10.980819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:10.998719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 902/1813] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:10.999983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.001653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.003699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.013387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 912/1814] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:11.014235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.016002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211bdf7-06fd-46cd-be8f-c4bfa22d28aa admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.028406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.029770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.030260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.030716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.031305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbede1-c6a3-4905-aa7d-fac5da1705ed service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.031843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.033998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.037421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.038142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.038434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.038753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.039752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.041661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.079039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 903/1815] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:10 2022] POST /identity/v3/users => generated 288 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:11.079039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.096153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 913/1816] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/projects => generated 401 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:11.097136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.107686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ec8b-c1a0-49e3-99c6-71ef30243abc admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.159193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:11.194859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c1373-d2bf-44e6-a783-d68c09a086b2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.199877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.200562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.200844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.201139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.201924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.203921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.206999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:11.253812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 914/1817] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:11.253812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.279276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.281308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.281684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.284726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.292079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4ee4a-0f2e-44fe-9ded-de1429f83eb4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.293501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.307614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.308880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.310092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.312046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.313037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.317446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.320966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.397965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 904/1818] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:11.401177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 915/1819] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:11.401998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.405039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.443206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b6950-0d42-41c6-a221-3b4f3981c62b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.449863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.450664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.451170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.451443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.452347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.453504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66530ba-4248-41b6-af1a-4175cbd8837a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.454852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.459743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.460858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.461357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.461830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.462772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.471115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 916/1820] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:11.471959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.478497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.490326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77a676-8c62-47cb-9502-20b85a42bd20 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.495839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 905/1821] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:11.496172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.496774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.497633 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.497925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.498527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] PATH_INFO: `/v3/projects/2be9e894b02e464fb2ba0ed559b74c9a/users/0d30ebaab1794c1e8194f22a058c99ec/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.508463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c32a8-d682-442d-8d7b-9bb4c0919680 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.515026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.515978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.516266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.516571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] PATH_INFO: `/v3/projects/e5c5651d2edc42528247d06a69c9f513/users/06b02ce3b73f477b8c7890cd63e408c3/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.517529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be9e894b02e464fb2ba0ed559b74c9a, user_id=0d30ebaab1794c1e8194f22a058c99ec, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.520443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.541813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c5651d2edc42528247d06a69c9f513, user_id=06b02ce3b73f477b8c7890cd63e408c3, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.549388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6364502-ae2b-485a-98a1-550cf0fa3f8b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.564346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 917/1822] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:11 2022] PUT /identity/v3/projects/2be9e894b02e464fb2ba0ed559b74c9a/users/0d30ebaab1794c1e8194f22a058c99ec/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:11.566750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.587719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 906/1823] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:11 2022] PUT /identity/v3/projects/e5c5651d2edc42528247d06a69c9f513/users/06b02ce3b73f477b8c7890cd63e408c3/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:11.588949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f80-e83e-4a13-8982-29db33e26092 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.589246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f80-e83e-4a13-8982-29db33e26092 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.590265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f80-e83e-4a13-8982-29db33e26092 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.593122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 907/1824] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:41:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:41:11.594762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd039-78f3-49b4-a390-c1e150f753d9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.595082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd039-78f3-49b4-a390-c1e150f753d9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.595354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd039-78f3-49b4-a390-c1e150f753d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.631443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c21fd039-78f3-49b4-a390-c1e150f753d9 None None] MFA Rules not processed for user `0d30ebaab1794c1e8194f22a058c99ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:11.632703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c21fd039-78f3-49b4-a390-c1e150f753d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:11.690841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fda3e9-49ff-45e6-87cf-9e83a2d8e4ab admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.755149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 908/1825] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:11.755149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.755149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.755149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.783591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.784568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.785070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.785662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.786605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.788715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.789107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] MFA Rules not processed for user `06b02ce3b73f477b8c7890cd63e408c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:11.791763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:11.803044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 918/1826] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:11.804369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.815768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99a4e9-4386-4b92-a0bd-64445ae707c2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.821471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:11.822525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.825590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.826290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.827303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:11.829923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b48a2-8038-4ada-b224-ad9a38614f9c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:11.870391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 919/1827] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:11.872275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:11.872647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:11.872983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:11.887899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:11.894696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:11.900060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:11.901100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 909/1828] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:11.901996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:11.916694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f1f0-d9c9-456b-ad54-ec519b1df9d0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:11.925177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:12.003851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 920/1829] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:12.005464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.014290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.016414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.017164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.017787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] PATH_INFO: `/v3/projects/53cdc6261a4542388cb90fc24bb6dd47/users/c6d40f8ab10b44adadac63c6cf33d710/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.031523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fad462-3842-46e2-85b9-0ef2b06bc767 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.042081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53cdc6261a4542388cb90fc24bb6dd47, user_id=c6d40f8ab10b44adadac63c6cf33d710, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.044957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.067636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.068364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.068650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.068925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.070103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.072098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.080719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 910/1830] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:11 2022] PUT /identity/v3/projects/53cdc6261a4542388cb90fc24bb6dd47/users/c6d40f8ab10b44adadac63c6cf33d710/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:12.081718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.130192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 921/1831] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:12.131475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.134029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:12.155860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f34e4-fc62-4422-83a4-fcc3c73cc88b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.283231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbf520-2658-44e3-95e5-4d015219dd1d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.298002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.299024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.299024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.299420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.300139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.305858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa268d68-6576-44b2-94e9-56e36b02fe24 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.326439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.327811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.327811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.328193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.339225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.341482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.369328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 911/1832] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/projects => generated 375 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:12.372932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.373228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.373517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.374005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 922/1833] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:12.377217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.401986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c26d-b2fe-430c-9182-ecb76336732d service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.428333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] MFA Rules not processed for user `c6d40f8ab10b44adadac63c6cf33d710`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:12.429989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:12.526561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.527790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.528288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.528763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.529753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.531775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.548415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 923/1834] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:12.549267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.569536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:12.578000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 912/1835] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:12.580935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d3067-b040-410e-80ec-ebe4f5ffdcc4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.583097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.585131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:12.590974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.592244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.592894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.593240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.594417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.599582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.605054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4101ad-b8cb-480e-8fd1-ea40bef409bd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.613535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.614805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.615338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.615690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] PATH_INFO: `/v3/projects/2c460e2af02e4a98830f7240d4e4d4f3` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.631043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c460e2af02e4a98830f7240d4e4d4f3)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.633435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.641611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 924/1836] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:12.647184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.663409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:12.677552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a29a93-89d2-444a-8544-1d9fff5e793a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.685987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.687150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.687474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.687794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] PATH_INFO: `/v3/projects/c799aed3c639447993c8c8e39a754534/users/e3ce18beef2845289881c8e7aec3c542/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.712222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c460e2af02e4a98830f7240d4e4d4f3', 'request_id': 'req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:12.723182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c460e2af02e4a98830f7240d4e4d4f3', 'request_id': 'req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:12.724766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c799aed3c639447993c8c8e39a754534, user_id=e3ce18beef2845289881c8e7aec3c542, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.727789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.736191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c460e2af02e4a98830f7240d4e4d4f3', 'request_id': 'req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:12.750961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 925/1837] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:12 2022] PUT /identity/v3/projects/c799aed3c639447993c8c8e39a754534/users/e3ce18beef2845289881c8e7aec3c542/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:12.751868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.753543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] The token cache is being invalidate because project 2c460e2af02e4a98830f7240d4e4d4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:12.754116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:12.755454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:12.757284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 913/1838] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:12 2022] DELETE /identity/v3/projects/2c460e2af02e4a98830f7240d4e4d4f3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:12.758330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.760131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:12.879279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c68a2-ec2d-49b8-b0d6-ccd9e1d83e5d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.886151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.886878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.887557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.888154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.888711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.890643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.894310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb5d18-39b0-47a3-8c13-de4f17df2363 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.902350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 914/1839] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:12.903613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:12.905594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.906304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:12.906692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.907219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.907674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.908572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.911074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.921957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d400a7-f031-4e9e-9d00-6e8f7c66a291 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:12.928156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:12.928920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.929274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.929748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.930550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:12.933441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ecbcf-4449-43e6-b8f7-a8277fc2c62b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:12.945993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 915/1840] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:12.947429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:12.947782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:12.948159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:12.953187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 926/1841] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/projects => generated 401 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:12.954293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.000802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:13.002164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.080392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414bbe-b1a2-403b-a7fe-8db5079afca5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.085563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.086292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.086584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.086862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.087872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.095237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587e2d-6c47-454f-ba75-29d137b7bab1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.114265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 927/1842] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:13.114265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e107b-67cf-4afc-825c-b663c665f27c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.114265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e107b-67cf-4afc-825c-b663c665f27c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.114265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e107b-67cf-4afc-825c-b663c665f27c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.114843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 928/1843] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:41:13.115636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e107b-67cf-4afc-825c-b663c665f27c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.131769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e107b-67cf-4afc-825c-b663c665f27c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.145112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.146072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.146368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.147922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] PATH_INFO: `/v3/projects/c799aed3c639447993c8c8e39a754534/users/e3ce18beef2845289881c8e7aec3c542/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.154140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c799aed3c639447993c8c8e39a754534, user_id=e3ce18beef2845289881c8e7aec3c542, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.155951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 916/1844] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:13.156968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.157241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.158815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.173007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 929/1845] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:13 2022] PUT /identity/v3/projects/c799aed3c639447993c8c8e39a754534/users/e3ce18beef2845289881c8e7aec3c542/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:13.173898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.174852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d73e7-8270-46f3-a900-1bd6a5fec8f6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.180600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:13.253465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:13.281319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f10a7-2d83-4694-81d6-b568c251b90b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.288813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.289984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.290478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.291431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.291880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.292308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.293156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.298207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca3ff2-d233-4073-8006-ad97faaae8e6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.299505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.299806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] PATH_INFO: `/v3/projects/ea9a8b4248dd4aeeaf8227ac6250b4c1/users/7a8b0456c4d34f34a58c73c443bc776a/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.337674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a8b4248dd4aeeaf8227ac6250b4c1, user_id=7a8b0456c4d34f34a58c73c443bc776a, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.342266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1ca9b-77f1-4477-8474-3a6627eaae92 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.343461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 917/1846] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/users => generated 307 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:13.345208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce53c-1494-467b-8d9c-5e24a67847f1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.345499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce53c-1494-467b-8d9c-5e24a67847f1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.345841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ce53c-1494-467b-8d9c-5e24a67847f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.367865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-cd1ce53c-1494-467b-8d9c-5e24a67847f1 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:13.369441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cd1ce53c-1494-467b-8d9c-5e24a67847f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.376320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 930/1847] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:13 2022] PUT /identity/v3/projects/ea9a8b4248dd4aeeaf8227ac6250b4c1/users/7a8b0456c4d34f34a58c73c443bc776a/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:13.377737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.378032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.378314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.398378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:13.399673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.498563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 918/1848] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:13.500666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.502308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.503090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.506655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 931/1849] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:13.508376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.524951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5caac0d-612a-420d-9488-f4a82df14483 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.537246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.538040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.538328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.538602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.539370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.549772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] MFA Rules not processed for user `e3ce18beef2845289881c8e7aec3c542`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:13.551132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.553157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65367a-38ac-4cf0-aa5c-15d3fd74393b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.573287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 932/1850] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:13.573287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.573562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.573642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.603738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] MFA Rules not processed for user `7a8b0456c4d34f34a58c73c443bc776a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:13.605003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.647480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 919/1851] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:13.648412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.665486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd0ca3-c7cd-4a9d-8997-cf0684a514c8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.673287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.674221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.674757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.675214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.676194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:13.676729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.678859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.687856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 933/1852] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:13.688944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.692305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.692769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:13.692987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 920/1853] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:13.694343 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.706807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a787-ac77-44f3-9e97-84a060970ffb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.708733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ddf3e-f16b-4ea0-bcfa-61aea64cbb39 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.712896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.714094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.714410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.714680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.714797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] PATH_INFO: `/v3/projects/45d8905d06354ee888e3cba557c59fc1/users/a19d2b81d905465da1f4715bb28a894f/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.715424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.715704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.715979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.726547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.728287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.731621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d8905d06354ee888e3cba557c59fc1, user_id=a19d2b81d905465da1f4715bb28a894f, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.734771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.739884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 934/1854] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:13.745060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.747090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:13.751137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 921/1855] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:13 2022] PUT /identity/v3/projects/45d8905d06354ee888e3cba557c59fc1/users/a19d2b81d905465da1f4715bb28a894f/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:13.752196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.805409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:13.824721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4c6e-7195-4300-a2e7-bb96915e7495 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.842165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7163538-e90e-4ce9-93ae-650cd6f5500d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.847845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.848651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.849063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.849439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.850234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.852338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.890046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.890786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.891136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.891417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.896490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 922/1856] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:13.896490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.974613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.976563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:13.985183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a2c3-57d6-4f50-816d-caa51c877529 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:13.990093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:13.990914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:13.991356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:13.991732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:13.992412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 935/1857] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:13.992701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:13.993325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:13.995146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.010507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bdefa-4a5b-4661-8b1e-cd28cf8ad1c9 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.018902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.020187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.020410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.020798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.022321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.024423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.039328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 923/1858] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:13 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:14.044383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.044675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 936/1859] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:14.045582 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.057500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dae55c-8796-46a6-9320-e6bd6e0058b4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.132809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.133659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.134034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.134407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.135250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.137639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.143370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:14.162504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bd5e8-d08e-46ee-8b6e-a1b7f936986c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.168197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.169519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.170012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.170460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.171513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.174108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 937/1860] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:14.174457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.174890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.187872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae31bc5-793d-41c5-b19e-bbac95c39c6b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.195670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.198211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.198496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.198770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] PATH_INFO: `/v3/projects/45d8905d06354ee888e3cba557c59fc1/users/a19d2b81d905465da1f4715bb28a894f/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.201970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d8905d06354ee888e3cba557c59fc1, user_id=a19d2b81d905465da1f4715bb28a894f, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.204300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 924/1861] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:14.205395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.209042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.226014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 938/1862] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:14 2022] PUT /identity/v3/projects/45d8905d06354ee888e3cba557c59fc1/users/a19d2b81d905465da1f4715bb28a894f/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:14.226541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.237627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:14.288188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f624d9-cebe-43e4-a469-86616bf3f838 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.311677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ee687-596b-4041-bde3-62171ef54393 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.320851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.321621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.321970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.322325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.323167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.326388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.337599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 939/1863] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:14.338464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.351939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.352688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.352877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.353150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.353436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280346da-754d-424e-b65a-f33fd9d1abd4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.354995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:14.359282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.359954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.360279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.360549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.361685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.362912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6cf66-7ffb-4689-9f9c-c4f6cbab66df admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.375634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 940/1864] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:14.377005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.377394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.377802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.411123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] MFA Rules not processed for user `a19d2b81d905465da1f4715bb28a894f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:14.412353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:14.419624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.421161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.430783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 925/1865] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:14.431713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.442979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589c493-233c-4e16-aaf4-f645545c62b8 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.448009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.448750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.449094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.449430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.450251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.452077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.468568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 926/1866] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:14.468568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.477042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614863a-378a-46a8-98ab-436cfc262a45 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.483089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.484088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.484436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.484808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] PATH_INFO: `/v3/projects/100e8ff1b0e649dc94a3b1897de6a931/users/a8d69a72493044f996f689a714ba8493/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.506759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100e8ff1b0e649dc94a3b1897de6a931, user_id=a8d69a72493044f996f689a714ba8493, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.507881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 941/1867] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:14.511083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.516310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.531511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 927/1868] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:14 2022] PUT /identity/v3/projects/100e8ff1b0e649dc94a3b1897de6a931/users/a8d69a72493044f996f689a714ba8493/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:14.532567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.533236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425fe7-bd5a-4b99-984d-8cedadfb5a2a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.540743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:14.581159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:14.610554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.612253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.612253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.612253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] PATH_INFO: `/v3/projects/7af8d58be0e542ec8ab871f6a5dfd81e/users/ff6be83c379c4704975183e7ea3f1b9c/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.630945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af8d58be0e542ec8ab871f6a5dfd81e, user_id=ff6be83c379c4704975183e7ea3f1b9c, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.639220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.666952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 942/1869] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:14 2022] PUT /identity/v3/projects/7af8d58be0e542ec8ab871f6a5dfd81e/users/ff6be83c379c4704975183e7ea3f1b9c/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:14.667918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.677540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8197b6-afdb-47af-9f38-f8aecd66d779 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.680370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:14.755220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.756453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.756746 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.757399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.784543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7f620-4afc-4a4c-a04b-3d74d2e87fd3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.790981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.791667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.791953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.792227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.793489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.796038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77281956-5daa-41b2-ab48-4562f5904246 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.808080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 943/1870] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:14.813681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.813976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.814263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.838201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.841334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.852125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] MFA Rules not processed for user `ff6be83c379c4704975183e7ea3f1b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:14.853333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 928/1871] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:14.853447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:14.867649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.896120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27430c1-637b-4c7a-a0f5-7d2bd981f214 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.923308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:14.925290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.925588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.926000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] PATH_INFO: `/v3/projects/100e8ff1b0e649dc94a3b1897de6a931/users/a8d69a72493044f996f689a714ba8493/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.929549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100e8ff1b0e649dc94a3b1897de6a931, user_id=a8d69a72493044f996f689a714ba8493, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:14.932971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141283b-4cef-4e55-8d91-9685c335b1d0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:14.947689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 929/1872] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:14 2022] PUT /identity/v3/projects/100e8ff1b0e649dc94a3b1897de6a931/users/a8d69a72493044f996f689a714ba8493/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:14.959098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:14.959469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:14.959998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:14.985435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 944/1873] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:14.985855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:14.998401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a525769-869e-4e82-95ae-b400a60ca7ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:14.999899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] MFA Rules not processed for user `a8d69a72493044f996f689a714ba8493`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:15.001337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.065796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:15.071659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.073355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.073748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.074213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.075242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.077663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.095644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 930/1874] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:15.099977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.109587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 945/1875] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:14 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:15.110328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.113742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5fad7-516e-4b8b-8d3d-a56c27fbe9df None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.136147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:15.175118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.176029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.176347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.176625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] PATH_INFO: `/v3/users/0d30ebaab1794c1e8194f22a058c99ec` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.177830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d30ebaab1794c1e8194f22a058c99ec)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.179834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.180942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8c57-e910-4e30-af8b-32578c076893 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.184928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.185654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.186010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.186295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.188209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.232678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d30ebaab1794c1e8194f22a058c99ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:15.233442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d30ebaab1794c1e8194f22a058c99ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:15.242358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d30ebaab1794c1e8194f22a058c99ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:15.250453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d30ebaab1794c1e8194f22a058c99ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:15.252344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.254049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.263968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d30ebaab1794c1e8194f22a058c99ec'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:15.264195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 946/1876] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:15.264893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.273165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 931/1877] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:15 2022] DELETE /identity/v3/users/0d30ebaab1794c1e8194f22a058c99ec => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:15.273165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.274541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.281715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3feb-6780-48da-b984-3915f590bb30 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.336987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5881a4a-272f-4cd0-828a-8026e51754c6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.341043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.341908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.342299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.342737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.345161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.350403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.351430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.351725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.352465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.354151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.357349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad907d-269f-4903-8bca-29e80585c5f7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.369562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:15.395293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 947/1878] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:15 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:15.395293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.395656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.395942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.417754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:15.419234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.429156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 948/1879] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:15.430101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.431953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.434921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.436489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.445649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb902-1ddf-4530-a6be-a2dafa4371e2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.448715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 932/1880] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:15.449870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.451328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.451577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.452466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.452878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.453149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.454114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.456093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.470343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 949/1881] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:15.474396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.489912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9eacf-9996-4b57-81e3-a5e0c12d2211 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.495313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.496323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.497001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.497312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] PATH_INFO: `/v3/projects/750d4c845896413fabb7b776649a3e60/users/993d77f6faf5400eb5e39e4fac625358/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.517943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d4c845896413fabb7b776649a3e60, user_id=993d77f6faf5400eb5e39e4fac625358, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.520983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.551882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 950/1882] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:15 2022] PUT /identity/v3/projects/750d4c845896413fabb7b776649a3e60/users/993d77f6faf5400eb5e39e4fac625358/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:15.555805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.561406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e452ced-623f-4f10-820c-2fe2e926b188 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.587410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:15.644831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.645585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.645942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.646477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.711032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b81ad-1718-4436-9a0a-ba7fece48974 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.723474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.724376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.724897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.725409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.726163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.728016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.742614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 951/1883] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:15.747665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:15.764655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70820-c4e6-419b-8ee5-0049b8983cfc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:15.768245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.770124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76af4f-b564-4807-94ab-a086761147a1 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.772599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:15.774206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.774576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.774952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] PATH_INFO: `/v3/projects/750d4c845896413fabb7b776649a3e60/users/993d77f6faf5400eb5e39e4fac625358/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.779223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d4c845896413fabb7b776649a3e60, user_id=993d77f6faf5400eb5e39e4fac625358, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:15.782523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:15.788740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 933/1884] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:15.801914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 952/1885] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:15 2022] PUT /identity/v3/projects/750d4c845896413fabb7b776649a3e60/users/993d77f6faf5400eb5e39e4fac625358/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:15.813211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:15.813437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:15.813751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:15.850365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] MFA Rules not processed for user `993d77f6faf5400eb5e39e4fac625358`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:15.851902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:15.896989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:16.037732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 934/1886] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:16.065658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:16.067375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:16.178342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c97c4c-2c46-4cab-a00c-68e451fb9f15 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:16.187013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:16.188991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:16.189392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:16.189807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:16.216520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:16.218878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:16.231311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 953/1887] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:16.730436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:16.812099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67882-475b-4483-aee1-a4a7c891269e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:16.817058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:16.818048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:16.818432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:16.818858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] PATH_INFO: `/v3/users/ff6be83c379c4704975183e7ea3f1b9c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:16.820243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6be83c379c4704975183e7ea3f1b9c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:16.822325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:16.882092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6be83c379c4704975183e7ea3f1b9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.882866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:16.883119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6be83c379c4704975183e7ea3f1b9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.884955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:16.898469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6be83c379c4704975183e7ea3f1b9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.901224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b3db6-e23e-492f-99af-09b6b10a265f service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:16.907603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:16.908485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:16.908771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:16.909037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] PATH_INFO: `/v3/projects/2be9e894b02e464fb2ba0ed559b74c9a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:16.910339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be9e894b02e464fb2ba0ed559b74c9a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:16.911762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6be83c379c4704975183e7ea3f1b9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.912282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:16.925052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2e3dfd63-361e-4e9f-a316-33935796d553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6be83c379c4704975183e7ea3f1b9c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.931385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 935/1888] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:16 2022] DELETE /identity/v3/users/ff6be83c379c4704975183e7ea3f1b9c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:16.935960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:16.936316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:16.936594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:16.955266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:16.956485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:16.962832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be9e894b02e464fb2ba0ed559b74c9a', 'request_id': 'req-ad5d4581-0884-4c2e-a417-2afea7aff5f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.970947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be9e894b02e464fb2ba0ed559b74c9a', 'request_id': 'req-ad5d4581-0884-4c2e-a417-2afea7aff5f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.977613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be9e894b02e464fb2ba0ed559b74c9a', 'request_id': 'req-ad5d4581-0884-4c2e-a417-2afea7aff5f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.987194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:16.989502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] The token cache is being invalidate because project 2be9e894b02e464fb2ba0ed559b74c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:16.989838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.990691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:16.995729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 954/1889] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:16 2022] DELETE /identity/v3/projects/2be9e894b02e464fb2ba0ed559b74c9a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:16.996671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:16.998142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.034680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:17.035922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 936/1890] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:17.041367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.042738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.111824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4581-0884-4c2e-a417-2afea7aff5f8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.123905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.124879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.126655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.126655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] PATH_INFO: `/v3/users/e3ce18beef2845289881c8e7aec3c542` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.126928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ce18beef2845289881c8e7aec3c542)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.129511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.158759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117115ef-e1a7-4dc4-9252-ed5750b26533 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.165089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.165810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.166232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.166424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.191795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.193331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.205956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 937/1891] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:17.217124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ce18beef2845289881c8e7aec3c542'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.217905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ce18beef2845289881c8e7aec3c542'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.226335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ce18beef2845289881c8e7aec3c542'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.241689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ce18beef2845289881c8e7aec3c542'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.250348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ce18beef2845289881c8e7aec3c542'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.259672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 955/1892] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:16 2022] DELETE /identity/v3/users/e3ce18beef2845289881c8e7aec3c542 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:17.586194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.587673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.608567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdda99-5255-4cb4-be5c-a555fcbdbc9a service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.632565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.634060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.682145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.683149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.683497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.683832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] PATH_INFO: `/v3/projects/7af8d58be0e542ec8ab871f6a5dfd81e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.685132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af8d58be0e542ec8ab871f6a5dfd81e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.687178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.705332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040764-37bb-47c5-b23b-f7d8cffb1073 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.709809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.710759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.711038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.711340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] PATH_INFO: `/v3/projects/c799aed3c639447993c8c8e39a754534` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.712509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c799aed3c639447993c8c8e39a754534)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.714589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.738636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af8d58be0e542ec8ab871f6a5dfd81e', 'request_id': 'req-044d5875-48d2-4316-830b-dbb603ad9ffe'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.744729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af8d58be0e542ec8ab871f6a5dfd81e', 'request_id': 'req-044d5875-48d2-4316-830b-dbb603ad9ffe'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.752684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af8d58be0e542ec8ab871f6a5dfd81e', 'request_id': 'req-044d5875-48d2-4316-830b-dbb603ad9ffe'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.760281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:17.763819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c799aed3c639447993c8c8e39a754534', 'request_id': 'req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.765483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] The token cache is being invalidate because project 7af8d58be0e542ec8ab871f6a5dfd81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:17.765959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.766641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.768374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 938/1893] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:17 2022] DELETE /identity/v3/projects/7af8d58be0e542ec8ab871f6a5dfd81e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:17.772583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.774297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.774670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c799aed3c639447993c8c8e39a754534', 'request_id': 'req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.782147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c799aed3c639447993c8c8e39a754534', 'request_id': 'req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.783583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:17.793824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] The token cache is being invalidate because project c799aed3c639447993c8c8e39a754534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:17.794213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.794860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-51d3e60a-9ce4-4db2-bdda-baf3a0499fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.796450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 956/1894] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:17 2022] DELETE /identity/v3/projects/c799aed3c639447993c8c8e39a754534 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:17.801339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14c23-eb40-4a54-8e90-f09658368bda None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.801619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14c23-eb40-4a54-8e90-f09658368bda None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.801892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14c23-eb40-4a54-8e90-f09658368bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.819631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ddb14c23-eb40-4a54-8e90-f09658368bda None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:17.820870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ddb14c23-eb40-4a54-8e90-f09658368bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.840445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5875-48d2-4316-830b-dbb603ad9ffe admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.883193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 957/1895] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:17.890953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.891383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.891721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.903106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.904010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.904291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.904569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] PATH_INFO: `/v3/users/993d77f6faf5400eb5e39e4fac625358` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.906184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] RBAC: Authorizing `identity:delete_user(user_id=993d77f6faf5400eb5e39e4fac625358)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.908455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.911721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:17.913222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:17.926075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 958/1896] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:17.930434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.941843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e79c07-7fe8-486c-b5ec-1746e9ca4e5d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.947829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.948610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.949004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.949263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.950255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:17.950569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.952249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:17.965017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 959/1897] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:17.969753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:17.971289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993d77f6faf5400eb5e39e4fac625358'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.977275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993d77f6faf5400eb5e39e4fac625358'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.986323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993d77f6faf5400eb5e39e4fac625358'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.986597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c97b3-347f-4da3-a146-c57b1bf63a93 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:17.992283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:17.992953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:17.993243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:17.993525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:17.994225 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:17.996064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993d77f6faf5400eb5e39e4fac625358'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:17.996258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.011715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993d77f6faf5400eb5e39e4fac625358'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.018874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 939/1898] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:17 2022] DELETE /identity/v3/users/993d77f6faf5400eb5e39e4fac625358 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:18.031104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 960/1899] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:17 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:18.032742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.034185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.080704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:18.113601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d460452-8376-4a7f-bd66-d4eca413226b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.180490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.180784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.181075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.181362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.184056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.186780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.216119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 940/1900] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:18 2022] POST /identity/v3/users => generated 281 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:18.222889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.237758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae7f31-2536-4bc2-b961-77eeb7e486a8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.252552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.253432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.253853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.254260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.255483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.257262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.274810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 961/1901] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:18.279574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.291233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a48d0-91e4-4a80-b302-822b2ac0058b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.296212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.296891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.297177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.297453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.298188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.300465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f5284-bb86-45ad-b65f-3a7b04375e57 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.312433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 941/1902] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:18.318464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.332556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066298d-204d-41d7-8fba-dc953f388a5c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.340066 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.341013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.341296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.341573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] PATH_INFO: `/v3/projects/ad310f73bc2b4d35b31d5204c99e1c28/users/e08f9afd897a49e896e0534fe9b2c0be/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.363919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad310f73bc2b4d35b31d5204c99e1c28, user_id=e08f9afd897a49e896e0534fe9b2c0be, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.366595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.384184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 962/1903] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:18 2022] PUT /identity/v3/projects/ad310f73bc2b4d35b31d5204c99e1c28/users/e08f9afd897a49e896e0534fe9b2c0be/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:18.394968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478505c3-9c7a-4778-995d-3802c16879db None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.395427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478505c3-9c7a-4778-995d-3802c16879db None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.396369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478505c3-9c7a-4778-995d-3802c16879db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.427825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-478505c3-9c7a-4778-995d-3802c16879db None None] MFA Rules not processed for user `e08f9afd897a49e896e0534fe9b2c0be`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:18.429488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-478505c3-9c7a-4778-995d-3802c16879db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.463544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.466068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.548222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 942/1904] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:18.592818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478505c3-9c7a-4778-995d-3802c16879db None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.595010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-478505c3-9c7a-4778-995d-3802c16879db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.610684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:18.620214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630250d-d575-4629-89ab-90156da582ef admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.625186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.626437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.626782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.629435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] PATH_INFO: `/v3/projects/750d4c845896413fabb7b776649a3e60` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.631853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750d4c845896413fabb7b776649a3e60)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.634528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.697041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:18.702381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478505c3-9c7a-4778-995d-3802c16879db None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.722076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750d4c845896413fabb7b776649a3e60', 'request_id': 'req-f539043a-7211-4b21-95d7-2e24e3abfb28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.734089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750d4c845896413fabb7b776649a3e60', 'request_id': 'req-f539043a-7211-4b21-95d7-2e24e3abfb28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.746293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750d4c845896413fabb7b776649a3e60', 'request_id': 'req-f539043a-7211-4b21-95d7-2e24e3abfb28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.761918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] The token cache is being invalidate because project 750d4c845896413fabb7b776649a3e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:18.762250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.763274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f539043a-7211-4b21-95d7-2e24e3abfb28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.765170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 963/1905] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:18 2022] DELETE /identity/v3/projects/750d4c845896413fabb7b776649a3e60 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:18.770275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50910ac6-ba18-445c-9a3e-d22c4ac6d72e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.770568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50910ac6-ba18-445c-9a3e-d22c4ac6d72e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.770845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50910ac6-ba18-445c-9a3e-d22c4ac6d72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.790321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.791029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.791358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.791634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.793616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.817372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-50910ac6-ba18-445c-9a3e-d22c4ac6d72e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:18.818992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-50910ac6-ba18-445c-9a3e-d22c4ac6d72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.877329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.878778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.880783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 964/1906] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:18.888701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.888864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 943/1907] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:18.888953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.889193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.902762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.904259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.908157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:18.909337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:18.917232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 965/1908] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:18.917761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e09e-e6ba-4569-b14f-a205ce1b4c52 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.921457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.922775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.923691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.923985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.924262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] PATH_INFO: `/v3/users/e08f9afd897a49e896e0534fe9b2c0be` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.925491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08f9afd897a49e896e0534fe9b2c0be)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.927483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.930460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e7615-1b01-4466-9327-32bcafbf3cba None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.935313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.935964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.936248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.936527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.937253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:18.937561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.939184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.950960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 966/1909] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:18.955259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:18.966063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716545eb-9364-473d-bf72-30a722d7f6b0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:18.970935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:18.971625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:18.971911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:18.972190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:18.972936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:18.974835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:18.981973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08f9afd897a49e896e0534fe9b2c0be'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.982672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08f9afd897a49e896e0534fe9b2c0be'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.991872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08f9afd897a49e896e0534fe9b2c0be'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:18.999496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08f9afd897a49e896e0534fe9b2c0be'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.006081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 967/1910] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:18 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:19.007094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.013234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e6cde7f-a0f0-40a3-a62f-a7222fb88a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08f9afd897a49e896e0534fe9b2c0be'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.019831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 944/1911] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:18 2022] DELETE /identity/v3/users/e08f9afd897a49e896e0534fe9b2c0be => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:19.024354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.024716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.025050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.042165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:19.043478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.055212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:19.071654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:19.074780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb45b-d8b1-409a-a100-ebeec0c1aff9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.118591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 945/1912] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:19.137770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.139632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.146622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.146622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.146946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.147214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.148047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.150003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.179510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 968/1913] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:19.184099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.186975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.202127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e822a-cf12-47cd-9841-2fb514bbd911 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.207877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.208908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.209196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.209475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.210229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.212370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.220744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259e73-a852-4f7d-85ec-c9cfc8aa7747 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.226200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.227312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.227872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.228231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.231756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 969/1914] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:19.236098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.250126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.250126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.250299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ed41-9559-474e-851d-ee022ae0ee2b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.256272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.256962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.257250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.257587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.258255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.259918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2cb5e0-6156-48cb-87fc-764a9f13ed91 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.260633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 946/1915] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:19.275264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 970/1916] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:19.279152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.293393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a1887-4689-4fae-8510-b57f550e5c4b service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.300092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.301085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.301778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.302238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] PATH_INFO: `/v3/projects/3f5186080e9846b5a504ca6e6bb8ca2a/users/cef97dd03c6c43628dac43364d5fab6a/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.321042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5186080e9846b5a504ca6e6bb8ca2a, user_id=cef97dd03c6c43628dac43364d5fab6a, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.323948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.338575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 947/1917] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:19 2022] PUT /identity/v3/projects/3f5186080e9846b5a504ca6e6bb8ca2a/users/cef97dd03c6c43628dac43364d5fab6a/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:19.353724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.354085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.354481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.382066 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] MFA Rules not processed for user `cef97dd03c6c43628dac43364d5fab6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:19.383337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.462220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 971/1918] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:19.467624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.545992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a43f36-5100-461c-9197-c5ceac9dfe8f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.550793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.558367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.558367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.558367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.558367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.560014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.560649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.561623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.596941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 948/1919] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:19.597095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b3fce-93c3-4587-bc94-5d4ff3b596aa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.597757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.639831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:19.648641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:19.666507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb1a6-5534-46ab-985e-7384afbac4c8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.670297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.671243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.671407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.671639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.671959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] PATH_INFO: `/v3/projects/ad310f73bc2b4d35b31d5204c99e1c28` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.672084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.672369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.672651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.673210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad310f73bc2b4d35b31d5204c99e1c28)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.673328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.675177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.675294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.699512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 949/1920] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:19.700747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.711656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c3a2c-245d-4d32-81f9-5e7e72e5b32b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.717156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.717893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.718186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.718472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.719443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.720885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.730840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad310f73bc2b4d35b31d5204c99e1c28', 'request_id': 'req-e4f6fd64-c917-4e33-b4da-4625b28c0b91'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.732198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 950/1921] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:19.736769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.745408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad310f73bc2b4d35b31d5204c99e1c28', 'request_id': 'req-e4f6fd64-c917-4e33-b4da-4625b28c0b91'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.748616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45608259-0137-4475-ba94-962bb920db5d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.752281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad310f73bc2b4d35b31d5204c99e1c28', 'request_id': 'req-e4f6fd64-c917-4e33-b4da-4625b28c0b91'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.763471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] The token cache is being invalidate because project ad310f73bc2b4d35b31d5204c99e1c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:19.763759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.764449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e4f6fd64-c917-4e33-b4da-4625b28c0b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:19.766049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 972/1922] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:19 2022] DELETE /identity/v3/projects/ad310f73bc2b4d35b31d5204c99e1c28 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:19.771163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55f81-6603-4bc7-a7f6-9be189a3aad4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.771499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55f81-6603-4bc7-a7f6-9be189a3aad4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.771782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55f81-6603-4bc7-a7f6-9be189a3aad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.790207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-84c55f81-6603-4bc7-a7f6-9be189a3aad4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:19.791467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-84c55f81-6603-4bc7-a7f6-9be189a3aad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.806810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.807755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.808042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.808323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] PATH_INFO: `/v3/projects/82039c59fd3448c4875ef6c1bf964ea1/users/1aaa12d578554ab8b9b5f307b2881af0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.825273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82039c59fd3448c4875ef6c1bf964ea1, user_id=1aaa12d578554ab8b9b5f307b2881af0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.827974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.846689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 951/1923] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:19 2022] PUT /identity/v3/projects/82039c59fd3448c4875ef6c1bf964ea1/users/1aaa12d578554ab8b9b5f307b2881af0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:19.847633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.849431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.867983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 973/1924] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:19.888306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.888692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.889035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.918002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:19.919338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:19.950596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7f9d-2a67-4d61-96cb-24218616861a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.957313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.958049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:19.958570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:19.958745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:19.959533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:19.961180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:19.972328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 952/1925] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:19.976505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:19.993704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5dd4a-155c-4f6e-9960-0bfbdac395f0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:19.994842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 974/1926] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:19.999393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:19.999902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.000360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.000590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.000874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] PATH_INFO: `/v3/projects/82039c59fd3448c4875ef6c1bf964ea1/users/1aaa12d578554ab8b9b5f307b2881af0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.014229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913e398-87e4-4adf-9ac9-89d3e2434667 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.017233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82039c59fd3448c4875ef6c1bf964ea1, user_id=1aaa12d578554ab8b9b5f307b2881af0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.020471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad48f5-66a3-4e6a-a378-13db7212d873 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.023625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.024813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.025466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.025899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.026946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:20.027299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.029120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.033154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 953/1927] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:19 2022] PUT /identity/v3/projects/82039c59fd3448c4875ef6c1bf964ea1/users/1aaa12d578554ab8b9b5f307b2881af0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:20.041603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 975/1928] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:20.043658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.044038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.044393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.046593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.056618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef363b-5521-4315-928e-0a58e2d1b5fd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.061885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:20.075265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] MFA Rules not processed for user `1aaa12d578554ab8b9b5f307b2881af0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:20.076558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:20.085678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:20.120006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.120675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.120962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.121242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.121984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.124008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.149351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 954/1929] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:20.153557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 976/1930] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:20 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:20.154938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.160716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.162403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:20.165703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa81c11-c178-403e-bb54-a4f4d53f767c None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.225268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8349-82e7-4636-a02d-8ee9635bb92b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.228913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.229568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.229677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.229844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.230114 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.230281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.230551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.230847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.231582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.233501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.255677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 955/1931] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:20 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:20.259637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.270417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d375f-725e-4cd2-b9e6-6d35e4499c70 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.276672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.277598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.278044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.278706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.279678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.281878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.292932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 956/1932] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:20.297403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.300068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.301518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.307566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3d5f-5574-4b0c-9a64-e58cfb174dc2 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.312362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.313052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.313336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.313617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.314350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.316112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 977/1933] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:20.324933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.340403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 957/1934] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:20.345515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.360800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981fb60-3f4a-4387-8fbc-e2ba208f758e service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.366654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.367826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.368277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.368780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] PATH_INFO: `/v3/projects/f0d9a5965b294b2ea2648367ac22befa/users/f037454e6b7e413ba98bcfe970548066/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.376222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.378075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:20.386562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d9a5965b294b2ea2648367ac22befa, user_id=f037454e6b7e413ba98bcfe970548066, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.389297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54609d0-2d6e-4031-b998-74cb1b770f4b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.392202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907f5f8-516d-496d-b165-ff1a6de67638 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.396393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.397098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.397484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.397777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.400535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:20.407699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 978/1935] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:20 2022] PUT /identity/v3/projects/f0d9a5965b294b2ea2648367ac22befa/users/f037454e6b7e413ba98bcfe970548066/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:20.417667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.418027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.418307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.448366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] MFA Rules not processed for user `f037454e6b7e413ba98bcfe970548066`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:20.449689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:20.458254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:20.474756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.476261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.485372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 958/1936] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:20.519139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 979/1937] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:20.531179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:20.540010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631a77-d00c-4131-b6c1-730c1d2b91a0 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:20.549099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:20.593311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:20.593974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:20.594262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:20.594542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:20.603728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:20.605247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:20.615428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 959/1938] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:22.628780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:22.639457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0969b12-37ab-4b6d-a368-ba4d686f625b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:22.643907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:22.644566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:22.644835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:22.645109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:22.647148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:22.708501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:22.709960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a47ab-4f2e-4bb2-8f67-19bd01e721f1 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:22.719311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 980/1939] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:23.866408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:23.938452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab240ef9-7948-4abf-be07-8474f1749ba5 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:23.943240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:23.944504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:23.944504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:23.944731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] PATH_INFO: `/v3/users/cef97dd03c6c43628dac43364d5fab6a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:23.945875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef97dd03c6c43628dac43364d5fab6a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:23.947881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:24.016029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef97dd03c6c43628dac43364d5fab6a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.016702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef97dd03c6c43628dac43364d5fab6a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.025364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef97dd03c6c43628dac43364d5fab6a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.032944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef97dd03c6c43628dac43364d5fab6a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.045940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef97dd03c6c43628dac43364d5fab6a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.052069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 960/1940] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:23 2022] DELETE /identity/v3/users/cef97dd03c6c43628dac43364d5fab6a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:24.056654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:24.057153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:24.057420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:24.077561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:24.079100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:24.147659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:24.149227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:24.157123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 981/1941] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:24.166995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:24.168459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:24.190605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:24.214979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdbc98-c90f-4fc1-a6fc-22932afc0825 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:24.219322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:24.219985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:24.220272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:24.220550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:24.222334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:24.222480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:24.227366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5024128-5fb0-40f2-8e95-b0816d74572d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:24.230785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:24.231491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:24.231773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:24.232044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:24.241417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:24.242840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:24.254478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 982/1942] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:24.293405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:24.295305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:24.307384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 961/1943] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:24.554466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:24.556031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:24.569021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5bc46-f10c-4127-9882-cc8735222e58 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:24.573754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:24.574591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:24.575230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:24.575507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] PATH_INFO: `/v3/projects/3f5186080e9846b5a504ca6e6bb8ca2a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:24.576923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5186080e9846b5a504ca6e6bb8ca2a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:24.578841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:24.631258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5186080e9846b5a504ca6e6bb8ca2a', 'request_id': 'req-81cb668c-57a3-4395-9a52-8f46888ebf29'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.640487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5186080e9846b5a504ca6e6bb8ca2a', 'request_id': 'req-81cb668c-57a3-4395-9a52-8f46888ebf29'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.647245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5186080e9846b5a504ca6e6bb8ca2a', 'request_id': 'req-81cb668c-57a3-4395-9a52-8f46888ebf29'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.658060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] The token cache is being invalidate because project 3f5186080e9846b5a504ca6e6bb8ca2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:24.658358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.658980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.660774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 983/1944] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:24 2022] DELETE /identity/v3/projects/3f5186080e9846b5a504ca6e6bb8ca2a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:24.664825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:24.675773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63a51-619d-41da-ba4d-b8a1947557a5 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:24.747823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:24.748723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:24.749002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:24.749286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] PATH_INFO: `/v3/users/1aaa12d578554ab8b9b5f307b2881af0` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:24.750898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aaa12d578554ab8b9b5f307b2881af0)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:24.752932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:24.801366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aaa12d578554ab8b9b5f307b2881af0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.802022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aaa12d578554ab8b9b5f307b2881af0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.809608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aaa12d578554ab8b9b5f307b2881af0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.817089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aaa12d578554ab8b9b5f307b2881af0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.829383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e28b3d1a-2b7f-4498-9518-13e24b5200b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aaa12d578554ab8b9b5f307b2881af0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:24.836130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 962/1945] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:24 2022] DELETE /identity/v3/users/1aaa12d578554ab8b9b5f307b2881af0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:25.203228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.204866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:25.224013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb668c-57a3-4395-9a52-8f46888ebf29 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.284467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.285353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.285636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.285909 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] PATH_INFO: `/v3/projects/82039c59fd3448c4875ef6c1bf964ea1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.287077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82039c59fd3448c4875ef6c1bf964ea1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.289116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.339362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82039c59fd3448c4875ef6c1bf964ea1', 'request_id': 'req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:25.348310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82039c59fd3448c4875ef6c1bf964ea1', 'request_id': 'req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:25.355291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82039c59fd3448c4875ef6c1bf964ea1', 'request_id': 'req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:25.366752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] The token cache is being invalidate because project 82039c59fd3448c4875ef6c1bf964ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:25.367077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:25.367874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d2d0e8e4-5112-41d2-9968-3c77d3d1c968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:25.369541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 984/1946] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:25 2022] DELETE /identity/v3/projects/82039c59fd3448c4875ef6c1bf964ea1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:25.374858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.375415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.375534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.382929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:25.397514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:25.398723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:25.463910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 963/1947] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:25.474531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.474819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.475123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.482986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:25.498522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:25.500405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:25.509014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 985/1948] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:25.513961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.524190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b4c2-0277-4dbd-8519-5ee62ff059dd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.529039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.529778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.530131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.530509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.531495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:25.531814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.533459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.548148 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 964/1949] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:25.552677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.563184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ffbf2-591c-48fc-92c8-940a8686ba23 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.568790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.569463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.569745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.570021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.570719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.572691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.603282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 986/1950] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:25.607298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.617481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea54e1-8578-4106-af6c-9ad24a3bb170 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.678111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.678835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.679145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.679424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.680124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.682087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.705183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 965/1951] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:25.709255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.720828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc675-b519-41b2-8d81-2031a6fd6b60 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.726597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.727303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.727583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.727860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.728595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.730220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.746458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 987/1952] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:25.751301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.762464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b30ab-3032-4c32-8acb-5a8ff53fa02a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.767797 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.768477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.769997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.769997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.769997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.771408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca2959-95cb-4fc9-ab59-b4df157f2aaa admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.782794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 966/1953] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:25.787085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:25.797865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54a393-383b-49ff-8b42-ddb0915175ab admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:25.802968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:25.803984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.804273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.804548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] PATH_INFO: `/v3/projects/51ebb651176d4a319dfc694eb7eee157/users/8ba9ad219fb9468d944236ed88aa4ed7/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.820984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ebb651176d4a319dfc694eb7eee157, user_id=8ba9ad219fb9468d944236ed88aa4ed7, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:25.823722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:25.838420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 988/1954] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:25 2022] PUT /identity/v3/projects/51ebb651176d4a319dfc694eb7eee157/users/8ba9ad219fb9468d944236ed88aa4ed7/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:25.852706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:25.852996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:25.853275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:25.882949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] MFA Rules not processed for user `8ba9ad219fb9468d944236ed88aa4ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:25.884243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:25.945943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:25.952470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 967/1955] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:25.957109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.019948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592fe93-36bb-4835-aa6b-6e6a73ed400e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.025099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.025766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.026049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.026319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.027006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.029036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.060354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 989/1956] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:25 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:26.064532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.075763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d8b7-b8e5-482d-ade1-af05cbc16c9f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.134364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.135041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.135355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.135631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.136332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.138740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.161828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 968/1957] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:26 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:26.165677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.176824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f95445-c36c-417c-af73-692faf6e3704 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.182049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.182713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.182986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.183287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.184039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.185640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.196700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 990/1958] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:26.203292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.214088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42add-b849-4838-88a2-b417daff1b48 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.220561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.221438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.221752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.223456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] PATH_INFO: `/v3/projects/b623b840e7d74ba982bb6d71f0b789ff/users/527d8c071820491ea0b8cf37f6c2f403/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.246844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b623b840e7d74ba982bb6d71f0b789ff, user_id=527d8c071820491ea0b8cf37f6c2f403, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.250081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.268346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 969/1959] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:26 2022] PUT /identity/v3/projects/b623b840e7d74ba982bb6d71f0b789ff/users/527d8c071820491ea0b8cf37f6c2f403/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:26.273052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.288820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07fe97-ed39-4aca-b0d2-e1c7247606c9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.347311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.347968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.348249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.348539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.349377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.350983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c0b5-6379-4c67-bc4e-0aa4d0eb8338 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.361779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 991/1960] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:26.365857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.378540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02adb0-f6d2-48be-9835-2f5aad58b681 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.384230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.385203 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.385400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.385678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] PATH_INFO: `/v3/projects/b623b840e7d74ba982bb6d71f0b789ff/users/527d8c071820491ea0b8cf37f6c2f403/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.387840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b623b840e7d74ba982bb6d71f0b789ff, user_id=527d8c071820491ea0b8cf37f6c2f403, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.390517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.401071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 970/1961] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:26 2022] PUT /identity/v3/projects/b623b840e7d74ba982bb6d71f0b789ff/users/527d8c071820491ea0b8cf37f6c2f403/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:26.411597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.411895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.412196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.432316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:26.440020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] MFA Rules not processed for user `527d8c071820491ea0b8cf37f6c2f403`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:26.442113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:26.514228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 992/1962] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:26.525205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.527530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:26.588129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e0a17-88b6-4ceb-983a-e0a4fa281e4d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.592266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.592915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.593197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.593477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.602478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.603945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c05a-7038-4385-bda2-4777e14cc75a service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.614710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 971/1963] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:26.674076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:26.675531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:26.687991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0fb6b-c9a6-4646-994a-8729eeedad41 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:26.691654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:26.692305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:26.692590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:26.692868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:26.751176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:26.752561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c138-e56f-4cf3-b1cb-055c4c141a2b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:26.763173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 993/1964] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:27.594855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:27.595504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:27.595605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:27.599865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 972/1965] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:41:27.606468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:27.607599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:27.607890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:27.629629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:27.631221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:27.711226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 994/1966] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:41:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:27.723456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:27.725372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:27.743544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d98ed-7ea1-4d8e-b0b4-e983f4a46755 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:27.749146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:27.749796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:27.750078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:27.750353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:27.760093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:27.761613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede07e2-d896-47cb-98d1-9bff7963fb74 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:27.772232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 973/1967] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:28.589082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:28.590694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:28.663116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:28.711371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129d378-a034-4bc4-b46f-c0412de952f7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:28.717957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:28.718927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:28.719265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:28.719523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] PATH_INFO: `/v3/users/c6d40f8ab10b44adadac63c6cf33d710` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:28.720906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d40f8ab10b44adadac63c6cf33d710)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:28.724192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:28.794433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d40f8ab10b44adadac63c6cf33d710'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:28.795648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d40f8ab10b44adadac63c6cf33d710'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:28.826994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d40f8ab10b44adadac63c6cf33d710'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:28.847350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d40f8ab10b44adadac63c6cf33d710'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:28.858381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d40f8ab10b44adadac63c6cf33d710'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:28.881453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 995/1968] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:28 2022] DELETE /identity/v3/users/c6d40f8ab10b44adadac63c6cf33d710 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:28.883421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:28.883968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:28.883968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:28.904999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:28.906286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:28.945644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:29.001676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 974/1969] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:29.062272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:29.063929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:29.152170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419448b4-77e9-4689-9a02-c3f254a5fddb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:29.158007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:29.158996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:29.159448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:29.160781 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:29.194793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:29.197439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87df53-071d-4d3e-8b87-64c087c892f6 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:29.214235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 996/1970] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:29.704971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:29.706489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:29.724844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87946-c854-42c9-b7ba-2727fae96017 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:29.731776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:29.732700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:29.732987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:29.733274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] PATH_INFO: `/v3/users/f037454e6b7e413ba98bcfe970548066` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:29.735164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f037454e6b7e413ba98bcfe970548066)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:29.737415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:29.812129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f037454e6b7e413ba98bcfe970548066'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:29.812716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f037454e6b7e413ba98bcfe970548066'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:29.822144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f037454e6b7e413ba98bcfe970548066'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:29.841366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f037454e6b7e413ba98bcfe970548066'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:29.857653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f037454e6b7e413ba98bcfe970548066'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:29.868450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 975/1971] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:29 2022] DELETE /identity/v3/users/f037454e6b7e413ba98bcfe970548066 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:29.873134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:29.873612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:29.873698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:29.901195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:29.902549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:29.977497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:30.004888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 997/1972] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:30.018836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:30.020639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:30.041912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:30.043902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:30.072972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2706e-e84f-4431-88ff-da978d0f8f3c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:30.080137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:30.081033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:30.081322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:30.084681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] PATH_INFO: `/v3/projects/53cdc6261a4542388cb90fc24bb6dd47` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:30.086077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53cdc6261a4542388cb90fc24bb6dd47)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:30.088000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:30.179380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea492-98ef-4af3-a079-7ec8fb738515 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:30.199869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53cdc6261a4542388cb90fc24bb6dd47', 'request_id': 'req-45c6e283-c274-43b6-bdec-3bba65030896'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.211469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53cdc6261a4542388cb90fc24bb6dd47', 'request_id': 'req-45c6e283-c274-43b6-bdec-3bba65030896'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.229051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53cdc6261a4542388cb90fc24bb6dd47', 'request_id': 'req-45c6e283-c274-43b6-bdec-3bba65030896'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.254926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] The token cache is being invalidate because project 53cdc6261a4542388cb90fc24bb6dd47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:30.255867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.262428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.268862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 998/1973] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:30 2022] DELETE /identity/v3/projects/53cdc6261a4542388cb90fc24bb6dd47 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:30.273637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:30.275687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:30.281701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:30.282816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:30.283350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:30.285522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:30.288627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:30.334636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:30.403493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:30.405660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:30.418611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 976/1974] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:30.421758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6e283-c274-43b6-bdec-3bba65030896 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:30.427095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:30.428452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:30.428973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:30.431452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] PATH_INFO: `/v3/users/a19d2b81d905465da1f4715bb28a894f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:30.432993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19d2b81d905465da1f4715bb28a894f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:30.434799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:30.493687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19d2b81d905465da1f4715bb28a894f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.494364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19d2b81d905465da1f4715bb28a894f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.504013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19d2b81d905465da1f4715bb28a894f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.527295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19d2b81d905465da1f4715bb28a894f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.530498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:30.560009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19d2b81d905465da1f4715bb28a894f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.577731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 999/1975] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:30 2022] DELETE /identity/v3/users/a19d2b81d905465da1f4715bb28a894f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:30.813296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:30.814801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:30.839657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992caa4d-a439-468d-91e2-bfe8a0e3e8e9 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:30.918947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:30.920525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:30.921131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:30.921742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] PATH_INFO: `/v3/projects/f0d9a5965b294b2ea2648367ac22befa` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:30.924185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d9a5965b294b2ea2648367ac22befa)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:30.928207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:30.986023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d9a5965b294b2ea2648367ac22befa', 'request_id': 'req-85c93398-8847-43c5-9bf3-bf149e6c3d28'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:30.993331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d9a5965b294b2ea2648367ac22befa', 'request_id': 'req-85c93398-8847-43c5-9bf3-bf149e6c3d28'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.001265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d9a5965b294b2ea2648367ac22befa', 'request_id': 'req-85c93398-8847-43c5-9bf3-bf149e6c3d28'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.016275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.016977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] The token cache is being invalidate because project f0d9a5965b294b2ea2648367ac22befa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:31.017270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.017978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-85c93398-8847-43c5-9bf3-bf149e6c3d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.019680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 977/1976] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:30 2022] DELETE /identity/v3/projects/f0d9a5965b294b2ea2648367ac22befa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:31.021339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.025463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87104e-81b5-4e3b-be93-5a8060210ca5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.025756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87104e-81b5-4e3b-be93-5a8060210ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.026038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87104e-81b5-4e3b-be93-5a8060210ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.049348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2f87104e-81b5-4e3b-be93-5a8060210ca5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:31.050328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae5fb6-b10e-46c9-823a-2ece2506d3c6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.051255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2f87104e-81b5-4e3b-be93-5a8060210ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.123020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.123973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.124263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.124787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] PATH_INFO: `/v3/projects/45d8905d06354ee888e3cba557c59fc1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.126268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d8905d06354ee888e3cba557c59fc1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.128316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.134076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:31.159819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 978/1977] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:31.169387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.169605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.169893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.190721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:31.192066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.194303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d8905d06354ee888e3cba557c59fc1', 'request_id': 'req-eeeb3fce-8094-4fe5-a3d5-31715918d015'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.206692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d8905d06354ee888e3cba557c59fc1', 'request_id': 'req-eeeb3fce-8094-4fe5-a3d5-31715918d015'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.214900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d8905d06354ee888e3cba557c59fc1', 'request_id': 'req-eeeb3fce-8094-4fe5-a3d5-31715918d015'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.229038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] The token cache is being invalidate because project 45d8905d06354ee888e3cba557c59fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:31.229444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.230172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eeeb3fce-8094-4fe5-a3d5-31715918d015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.232088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1000/1978] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:31 2022] DELETE /identity/v3/projects/45d8905d06354ee888e3cba557c59fc1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:31.239172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085388-a8ce-4860-9fcc-fca71c916268 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.239493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085388-a8ce-4860-9fcc-fca71c916268 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.239819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085388-a8ce-4860-9fcc-fca71c916268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.275688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f5085388-a8ce-4860-9fcc-fca71c916268 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:31.277676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 979/1979] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:31.279231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5085388-a8ce-4860-9fcc-fca71c916268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.283793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.287413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1001/1980] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:31.294012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf393a-e7df-4970-b6e1-ab4abcd2142d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.295124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.295420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.295702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.299211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.299879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.300193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.300429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.301315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:31.301624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.303288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.313586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:31.314801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.315399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 980/1981] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:31.319937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.323201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1002/1982] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:31.327383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.330911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579913c6-98ee-40c6-91e0-40ae8d1e132b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.335633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.336132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a570f-649e-44bc-8263-7d66d9239b0a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.336392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.336682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.336961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.337680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.339636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.340685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.341324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.341605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.341876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.342593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:31.342903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.344554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.356404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1003/1983] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:31.361515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.368835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 981/1984] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:31.371692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165ebc-2858-4d65-9900-6761bc820f21 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.372918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.460883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9951ab-d310-4af6-8dc4-a52b0a76cd0a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.467018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:31.477160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.478028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.478315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.478595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.479536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.481984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.482142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.482574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.482892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.484817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.485526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.488433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.512764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 982/1985] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:31.516893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.541554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47e770-0ef5-4b68-9234-edd6343f16cb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.562094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.562789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.563125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.563415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.564172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.565109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1004/1986] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:31.566079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.569128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.589473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 983/1987] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:31.591800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.615140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfa597-7c74-4a84-bf7e-1263fc8c9c0e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.688367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:31.710854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246ac0-9c0f-484f-a81e-249ff5a38c4d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.721726 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.722457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.723655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.723936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.724644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.727679 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.743317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.744008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.744293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.744780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.745468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.747174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.772596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1005/1988] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:31 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:31.772596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.772760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 984/1989] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:31.772760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.773926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:31.788318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553427b-3429-4b5a-8a19-89da1714619c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.794365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.795414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.795810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.796090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] PATH_INFO: `/v3/projects/0d546952fa934a43a3443b91dfee479c/users/98826119296446c08dce72aff0a03c5d/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.798506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799f658-e8bf-46ff-acc0-5cb1e189bc24 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.814068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:31.814848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:31.815128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d546952fa934a43a3443b91dfee479c, user_id=98826119296446c08dce72aff0a03c5d, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.815227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:31.815475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] PATH_INFO: `/v3/users/8ba9ad219fb9468d944236ed88aa4ed7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:31.819089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.819374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba9ad219fb9468d944236ed88aa4ed7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:31.821333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:31.838355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 985/1990] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:31 2022] PUT /identity/v3/projects/0d546952fa934a43a3443b91dfee479c/users/98826119296446c08dce72aff0a03c5d/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:31.839157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.908076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba9ad219fb9468d944236ed88aa4ed7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.915622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba9ad219fb9468d944236ed88aa4ed7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.926200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba9ad219fb9468d944236ed88aa4ed7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.939233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba9ad219fb9468d944236ed88aa4ed7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.948147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba9ad219fb9468d944236ed88aa4ed7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:31.954259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f9404-6bd6-4091-92c4-0e5b6a77a8d0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:31.955787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1006/1991] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:31 2022] DELETE /identity/v3/users/8ba9ad219fb9468d944236ed88aa4ed7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:31.960434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:31.961402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.016685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:32.038618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.039537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.039849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aa195-aba9-465f-8cea-53c9c387439d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.040083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.040532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.041445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.043791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4bf1f-4308-4337-ae1c-008cce10b54b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.045866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.047080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.047529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.047946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] PATH_INFO: `/v3/users/06b02ce3b73f477b8c7890cd63e408c3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.049987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b02ce3b73f477b8c7890cd63e408c3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.052711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.058689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 986/1992] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:32.058689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.058689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.058689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.086892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] MFA Rules not processed for user `98826119296446c08dce72aff0a03c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:32.088168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.111160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b02ce3b73f477b8c7890cd63e408c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:32.111822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b02ce3b73f477b8c7890cd63e408c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:32.118748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b02ce3b73f477b8c7890cd63e408c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:32.129296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:32.132355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b02ce3b73f477b8c7890cd63e408c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:32.141186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1572e8a1-c253-4dfa-a0a2-6cb5d39387bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b02ce3b73f477b8c7890cd63e408c3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:32.148008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1007/1993] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:31 2022] DELETE /identity/v3/users/06b02ce3b73f477b8c7890cd63e408c3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:32.149334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7556-ef41-46c6-9534-516b7cae7a71 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.149626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7556-ef41-46c6-9534-516b7cae7a71 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.149903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7556-ef41-46c6-9534-516b7cae7a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.163044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 987/1994] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:32.163044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.163044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.172473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-674c7556-ef41-46c6-9534-516b7cae7a71 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:32.173670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-674c7556-ef41-46c6-9534-516b7cae7a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.175636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c4c18-38f9-49b3-976f-73e43a71197f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.235147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.235685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1008/1995] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:32.235862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.236031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.236308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.236982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.237085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.237274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.237551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.238708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.255869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 988/1996] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:32.256879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.258647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.258811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:32.260263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.269267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1009/1997] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:32.269877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.272684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.324529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b0653-9010-44f1-92c4-55e577281f1f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.328274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.328927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.329205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.329484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.334732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5066b9-e62c-426f-87c4-daa60badb433 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.341650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.342323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.342612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.342890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.353644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.355147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.360458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:32.370076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1010/1998] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:32.370964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.384745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93596fdb-08db-4ba5-b3e8-865943206514 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.391552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.391728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.392513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.392800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.393079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] PATH_INFO: `/v3/projects/59a2245edc014b7082696f35af04ba26/users/f0a202a9fc384d818f702763e1f5c0c1/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.393232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.403816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 989/1999] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:32.404836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.406316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.410887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a2245edc014b7082696f35af04ba26, user_id=f0a202a9fc384d818f702763e1f5c0c1, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.413830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1fcb9-d0e0-47a2-834e-93c1c13a5738 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.420995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e67b7-358a-4937-b87e-c17abd3a9101 service glance] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.425008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.425906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.426227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.426700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.437733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1011/2000] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:32 2022] PUT /identity/v3/projects/59a2245edc014b7082696f35af04ba26/users/f0a202a9fc384d818f702763e1f5c0c1/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:32.438838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.439491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.439777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.445172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.446801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.458421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:32.460046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.460766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 990/2001] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:32.558213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1012/2002] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:32.570712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.584979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c16d-567b-4de4-b91e-eee46ac7cace service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.601273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:32.653363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.653978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.654262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.654536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.665217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.666644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.679209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 991/2003] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:32.849310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.850888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.883128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc38e08-4f5b-433b-9ee3-29881e64bf7d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.890015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:32.891384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.891755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:32.892275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.892560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.892905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] PATH_INFO: `/v3/projects/e5c5651d2edc42528247d06a69c9f513` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.894076 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c5651d2edc42528247d06a69c9f513)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.896532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:32.907977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575653b-d828-4600-92d8-cced9d733095 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:32.917476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:32.919475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:32.919475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:32.919475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] PATH_INFO: `/v3/projects/51ebb651176d4a319dfc694eb7eee157` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:32.920632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ebb651176d4a319dfc694eb7eee157)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:32.923541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:33.030883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c5651d2edc42528247d06a69c9f513', 'request_id': 'req-0f913b0e-f4c2-4064-9645-22b9117de99f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.042655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c5651d2edc42528247d06a69c9f513', 'request_id': 'req-0f913b0e-f4c2-4064-9645-22b9117de99f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.054683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c5651d2edc42528247d06a69c9f513', 'request_id': 'req-0f913b0e-f4c2-4064-9645-22b9117de99f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.071663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ebb651176d4a319dfc694eb7eee157', 'request_id': 'req-7f7c5a83-2242-42a5-8045-e6651dc50702'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.087669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ebb651176d4a319dfc694eb7eee157', 'request_id': 'req-7f7c5a83-2242-42a5-8045-e6651dc50702'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.102316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] The token cache is being invalidate because project e5c5651d2edc42528247d06a69c9f513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:33.102602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.113715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.115600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ebb651176d4a319dfc694eb7eee157', 'request_id': 'req-7f7c5a83-2242-42a5-8045-e6651dc50702'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.118221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1013/2004] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:32 2022] DELETE /identity/v3/projects/e5c5651d2edc42528247d06a69c9f513 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:33.119144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:33.122954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:33.143262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] The token cache is being invalidate because project 51ebb651176d4a319dfc694eb7eee157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:33.143635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.144372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.147220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 992/2005] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:32 2022] DELETE /identity/v3/projects/51ebb651176d4a319dfc694eb7eee157 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:33.153823 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:33.155509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:33.260179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:33.267404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913b0e-f4c2-4064-9645-22b9117de99f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:33.275894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:33.276270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:33.276871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:33.277338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c5a83-2242-42a5-8045-e6651dc50702 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:33.277486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] PATH_INFO: `/v3/users/7a8b0456c4d34f34a58c73c443bc776a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:33.279335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8b0456c4d34f34a58c73c443bc776a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:33.281637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:33.282635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:33.283679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:33.283970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:33.284258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] PATH_INFO: `/v3/users/527d8c071820491ea0b8cf37f6c2f403` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:33.285698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=527d8c071820491ea0b8cf37f6c2f403)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:33.287960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:33.407258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8b0456c4d34f34a58c73c443bc776a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.408304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8b0456c4d34f34a58c73c443bc776a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.412878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:33.420697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '527d8c071820491ea0b8cf37f6c2f403'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.421223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8b0456c4d34f34a58c73c443bc776a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.422807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '527d8c071820491ea0b8cf37f6c2f403'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.438760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '527d8c071820491ea0b8cf37f6c2f403'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.453811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '527d8c071820491ea0b8cf37f6c2f403'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.456246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8b0456c4d34f34a58c73c443bc776a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.472656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c89ec60-33f4-4c33-b8a7-5e2f7aa46f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8b0456c4d34f34a58c73c443bc776a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.491608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1014/2006] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:33 2022] DELETE /identity/v3/users/7a8b0456c4d34f34a58c73c443bc776a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:33.496189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '527d8c071820491ea0b8cf37f6c2f403'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:33.552068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 993/2007] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:33 2022] DELETE /identity/v3/users/527d8c071820491ea0b8cf37f6c2f403 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:34.155578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.156138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.156138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.182963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:34.186207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.211826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] MFA Rules not processed for user `f0a202a9fc384d818f702763e1f5c0c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:34.213063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.304812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ec05-3efe-4300-911f-da4a49b278f8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:34.310902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:34.311844 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.312133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.312424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] PATH_INFO: `/v3/projects/ea9a8b4248dd4aeeaf8227ac6250b4c1` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.313728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9a8b4248dd4aeeaf8227ac6250b4c1)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:34.315828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:34.326416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1015/2008] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:34.327615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:34.329145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.347613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363a850-aa10-4109-a7c8-66ef94059a63 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:34.357346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:34.358319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.358607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.358885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] PATH_INFO: `/v3/projects/b623b840e7d74ba982bb6d71f0b789ff` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.363251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b623b840e7d74ba982bb6d71f0b789ff)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:34.365305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:34.381588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9a8b4248dd4aeeaf8227ac6250b4c1', 'request_id': 'req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.389893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9a8b4248dd4aeeaf8227ac6250b4c1', 'request_id': 'req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.403178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9a8b4248dd4aeeaf8227ac6250b4c1', 'request_id': 'req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.425684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] The token cache is being invalidate because project ea9a8b4248dd4aeeaf8227ac6250b4c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:34.426334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.427339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.429235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 994/2009] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:34 2022] DELETE /identity/v3/projects/ea9a8b4248dd4aeeaf8227ac6250b4c1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:34.430366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:34.432033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.456656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b623b840e7d74ba982bb6d71f0b789ff', 'request_id': 'req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.471376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b623b840e7d74ba982bb6d71f0b789ff', 'request_id': 'req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.479104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b623b840e7d74ba982bb6d71f0b789ff', 'request_id': 'req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.480549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:34.505755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] The token cache is being invalidate because project b623b840e7d74ba982bb6d71f0b789ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:34.506075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.507512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.509418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1016/2010] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:34 2022] DELETE /identity/v3/projects/b623b840e7d74ba982bb6d71f0b789ff => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:34.511527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:34.513155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.519730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:34.549264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a62e6-cf3e-49a6-9039-42b8e920fd49 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:34.596554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a07b2-8857-48e8-a7eb-7b90195dcc3c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:34.601310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:34.602196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.602496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.602903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] PATH_INFO: `/v3/users/a8d69a72493044f996f689a714ba8493` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.604050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d69a72493044f996f689a714ba8493)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:34.606402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:34.611949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:34.612686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.612973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.613251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.613955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:34.616116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5172dd-517b-4246-a446-b5320e7c4f6e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:34.654873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 995/2011] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:34 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:34.656942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571ebd-1686-4789-8adf-292f1268c1be None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.657582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571ebd-1686-4789-8adf-292f1268c1be None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.658115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05571ebd-1686-4789-8adf-292f1268c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.678162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-05571ebd-1686-4789-8adf-292f1268c1be None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:34.679534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-05571ebd-1686-4789-8adf-292f1268c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.680931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d69a72493044f996f689a714ba8493'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.682231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d69a72493044f996f689a714ba8493'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.689766 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d69a72493044f996f689a714ba8493'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.732432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d69a72493044f996f689a714ba8493'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.743584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d69a72493044f996f689a714ba8493'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:34.754105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1017/2012] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:34 2022] DELETE /identity/v3/users/a8d69a72493044f996f689a714ba8493 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:34.755974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:34.759221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.775251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 996/2013] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:34.784592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.784877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.785153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.822958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:34.824522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:34.928098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d8ae-829c-4a4f-86de-3c1d8ea7d3e8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:34.935288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:34.936387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:34.936783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:34.937189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:34.938019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:34.940357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:34.966692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:34.983096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1018/2014] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:34 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:34.983096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.011259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 997/2015] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:35.015816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.024854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf088-c9fe-4252-9647-b50cd753199c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.035790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d64cf0-9f1d-474a-80f0-8a2541f3e4e3 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.054234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.055597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.056651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.057357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.061838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.064356 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.066424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.070144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.070624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.071957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.080806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:35.081528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.086143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.091307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1019/2016] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:35.099186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.117904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 998/2017] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:35.123713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.129332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f5570-4f4c-41f5-9dfe-f4aea9f10da2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.142555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.143688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.143982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.144261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] PATH_INFO: `/v3/projects/79d50f4bd85a452e8c46056da9a06510/users/0d338967a4f2409d885a6330a68c6005/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.144771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50afd0-ed43-454c-8987-127cff2a25ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.161169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.162288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.164383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.165383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.168077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.173833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.189583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d50f4bd85a452e8c46056da9a06510, user_id=0d338967a4f2409d885a6330a68c6005, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.193089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.206132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:35.227535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1020/2018] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:35 2022] PUT /identity/v3/projects/79d50f4bd85a452e8c46056da9a06510/users/0d338967a4f2409d885a6330a68c6005/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:35.231570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.254936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 999/2019] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:35.257718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.364249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a717ab8-3e48-49c0-b225-6e6ad43a0648 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.373494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2631e9-3798-44d4-83bd-4e27f0cf74b8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.374313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.379624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.379624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.379624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.379624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.380562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.381231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.381533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.381803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.382032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.382537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.384547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.410332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1021/2020] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:35.411324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.413224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.426276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2437e44-a008-448d-a3f9-e317f5c45184 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.426799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1000/2021] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:35.427765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.441207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.442956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.443343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.443696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] PATH_INFO: `/v3/projects/100e8ff1b0e649dc94a3b1897de6a931` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.445372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100e8ff1b0e649dc94a3b1897de6a931)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.447378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcc259-5f9b-4602-8b0e-6da1d7fd259e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.447494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.454862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.456926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.457251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.457412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] PATH_INFO: `/v3/projects/79d50f4bd85a452e8c46056da9a06510/users/0d338967a4f2409d885a6330a68c6005/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.461160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d50f4bd85a452e8c46056da9a06510, user_id=0d338967a4f2409d885a6330a68c6005, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.465298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.482651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1001/2022] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:35 2022] PUT /identity/v3/projects/79d50f4bd85a452e8c46056da9a06510/users/0d338967a4f2409d885a6330a68c6005/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:35.483737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.516999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100e8ff1b0e649dc94a3b1897de6a931', 'request_id': 'req-d70b6134-44c9-4d1d-85a4-47302a73bac2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:35.525834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100e8ff1b0e649dc94a3b1897de6a931', 'request_id': 'req-d70b6134-44c9-4d1d-85a4-47302a73bac2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:35.537220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100e8ff1b0e649dc94a3b1897de6a931', 'request_id': 'req-d70b6134-44c9-4d1d-85a4-47302a73bac2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:35.551811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] The token cache is being invalidate because project 100e8ff1b0e649dc94a3b1897de6a931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:35.552162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:35.553677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d70b6134-44c9-4d1d-85a4-47302a73bac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:35.556295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1022/2023] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:35 2022] DELETE /identity/v3/projects/100e8ff1b0e649dc94a3b1897de6a931 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:35.562485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f5304-d5bb-47f6-9841-95814c5406df None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.562734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f5304-d5bb-47f6-9841-95814c5406df None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.563008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f5304-d5bb-47f6-9841-95814c5406df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.599842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6c5f5304-d5bb-47f6-9841-95814c5406df None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:35.601717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c5f5304-d5bb-47f6-9841-95814c5406df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.606839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b0ac6-6720-4899-b942-4b9ffd5c7b87 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.656823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:35.681390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.682297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.682534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.683013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.683842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.685231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1023/2024] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:35.692525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.693131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.693532 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.693989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.705064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1002/2025] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:35.709275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.710955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.718979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:35.721283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.726255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a81448-2409-4775-85d3-5d67f0a89b2a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.731155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1024/2026] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:35.732072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.732969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.733257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.734032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.734787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.735645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.737311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.740282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:35.745734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2215b-73c9-479d-b6be-cec4384f840c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.750953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.751668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.751934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.752207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.753055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:35.753260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.754909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1003/2027] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:35.755837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.756126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.768516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e315-db44-498d-bded-5617e1fa6d08 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.774045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.774997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.775374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.775722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] PATH_INFO: `/v3/projects/8715f0e0ec7f4875a347f9054479f4be/users/e49dde056aa546aab3c48af2992d6006/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.775903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1025/2028] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:35.776748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.787398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7f72a-d540-4b0c-8e35-0cb1771a3335 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.793028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.793183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8715f0e0ec7f4875a347f9054479f4be, user_id=e49dde056aa546aab3c48af2992d6006, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.793814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.794181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.794956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.795336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.797456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.798370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31caec1-1d86-4cfb-92a0-972021ec30a5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.815597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1004/2029] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:35 2022] PUT /identity/v3/projects/8715f0e0ec7f4875a347f9054479f4be/users/e49dde056aa546aab3c48af2992d6006/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:35.827691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.828111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.828883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.840202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1026/2030] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:35.844173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.859953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] MFA Rules not processed for user `e49dde056aa546aab3c48af2992d6006`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:35.861192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.915144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c19a-28bb-4c71-b7a0-b4a2f6764cd1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.923408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.924125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.924401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.924672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.925580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.927548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.945933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1005/2031] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:35.953940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1027/2032] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:35 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:35.958001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.959183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:35.967619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:35.971905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bbe01-fed1-41a0-b758-aa243a14a45a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:35.978894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:35.979373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:35.979667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:35.980608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:35.981784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:35.983308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:35.997122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1006/2033] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:36.000827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:36.016259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b3e68-81b8-4490-8207-6c26496494a2 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:36.021349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:36.022087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.022374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.023101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.023411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:36.025472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:36.039561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08681579-bfc4-4980-b95a-defdac033474 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:36.042058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1007/2034] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:36.042962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:36.046372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:36.048003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.048535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.049009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.054731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d2feb-29e5-416e-86ce-29e2cabd4cd5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:36.060711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:36.061674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.061960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.062186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:36.062295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] PATH_INFO: `/v3/projects/7df664c6311a4aefb792aa792ebc17fa/users/f9aced17fa4848ecb0d5be31b87e1e5a/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.063778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d405398-f2cc-4875-98ee-302ceb78ad10 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:36.074756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1028/2035] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:36.080472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df664c6311a4aefb792aa792ebc17fa, user_id=f9aced17fa4848ecb0d5be31b87e1e5a, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:36.083255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:36.098035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1008/2036] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:36 2022] PUT /identity/v3/projects/7df664c6311a4aefb792aa792ebc17fa/users/f9aced17fa4848ecb0d5be31b87e1e5a/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:36.103197 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e12c-93bd-473e-8878-eda9a74cb2fe None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.103553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e12c-93bd-473e-8878-eda9a74cb2fe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.103895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e12c-93bd-473e-8878-eda9a74cb2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.123611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c96e12c-93bd-473e-8878-eda9a74cb2fe None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:36.124926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c96e12c-93bd-473e-8878-eda9a74cb2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:36.160292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:36.194833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1029/2037] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:36.206729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:36.208928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:36.278725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494376-4871-45cd-ab1a-fc9206b1f385 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:36.283112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:36.283965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.284739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.284739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.300156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:36.301744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:36.322124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c7fe-fe69-433e-9262-ad6cd66674e7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.322416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c7fe-fe69-433e-9262-ad6cd66674e7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.322701 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c7fe-fe69-433e-9262-ad6cd66674e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.336758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1009/2038] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:36.357177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1c04c7fe-fe69-433e-9262-ad6cd66674e7 None None] MFA Rules not processed for user `0d338967a4f2409d885a6330a68c6005`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:36.358492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c04c7fe-fe69-433e-9262-ad6cd66674e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:36.472547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1030/2039] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:36.480224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:36.521417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:36.599773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686397-1257-4465-a047-8690493cf5b4 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:36.603643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:36.604356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:36.604633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:36.604924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:36.607113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:36.678023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:36.679632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:36.689348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1010/2040] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:37.496685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0217d1-e73a-42b3-a363-fa169010bdcb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:37.497614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0217d1-e73a-42b3-a363-fa169010bdcb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:37.497901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0217d1-e73a-42b3-a363-fa169010bdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:37.527950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aa0217d1-e73a-42b3-a363-fa169010bdcb None None] MFA Rules not processed for user `f9aced17fa4848ecb0d5be31b87e1e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:37.529866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aa0217d1-e73a-42b3-a363-fa169010bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:37.608673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1031/2041] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:37.619721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:37.630935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f274091-5613-4154-a81d-8c06ab3f3c31 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:37.635985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:37.636400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:37.636693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:37.637144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:37.647787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:37.649445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:37.660138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1011/2042] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:43.238515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:43.239010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:43.239177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:43.263814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] MFA Rules not processed for user `f9aced17fa4848ecb0d5be31b87e1e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:43.265097 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:43.295621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1032/2043] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:43.387431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:43.389323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:43.404743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284583a3-a566-4d12-8599-daacc828a6e2 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:43.409568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:43.410233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:43.410522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:43.410808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:43.433266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:43.434899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4712ea-4fe3-4fd1-9fc6-00f80ce43f6f service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:43.450424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1012/2044] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:53.254856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:53.443833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=51 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:55.927552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:55.929312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:55.947903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21bc8b-5a25-4e0b-b994-8c6276707b8a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:55.953247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:55.954251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:55.954529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:55.954855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] PATH_INFO: `/v3/users/90f6f184c562423ea57295465ec5b211` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:55.956273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f6f184c562423ea57295465ec5b211)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:55.958292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:56.026014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f6f184c562423ea57295465ec5b211'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.027202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f6f184c562423ea57295465ec5b211'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.039378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f6f184c562423ea57295465ec5b211'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.052728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f6f184c562423ea57295465ec5b211'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.062630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f6f184c562423ea57295465ec5b211'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.070810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1033/2045] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:55 2022] DELETE /identity/v3/users/90f6f184c562423ea57295465ec5b211 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:56.075393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:56.075858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:56.075987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:56.094514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:56.096043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:56.175997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1013/2046] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:56.190557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:56.192837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:56.272431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2819960-c7b2-4d24-a291-7d786775b9be admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:56.277069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:56.277739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:56.279422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:56.279941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:56.290333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:56.292085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:56.306887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1034/2047] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:56.622373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:56.623808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:56.642451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba6de6-1605-4d1b-ad97-b869ff09f3cf None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:56.647464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:56.648407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:56.648723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:56.648999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] PATH_INFO: `/v3/projects/9b80d169e3124b04b4a2f31017ad6eb3` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:56.651158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b80d169e3124b04b4a2f31017ad6eb3)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:56.653528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:56.706364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b80d169e3124b04b4a2f31017ad6eb3', 'request_id': 'req-b9c297d5-f9ad-448c-945e-4ead8a84627b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.713773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b80d169e3124b04b4a2f31017ad6eb3', 'request_id': 'req-b9c297d5-f9ad-448c-945e-4ead8a84627b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.723570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b80d169e3124b04b4a2f31017ad6eb3', 'request_id': 'req-b9c297d5-f9ad-448c-945e-4ead8a84627b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.736753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] The token cache is being invalidate because project 9b80d169e3124b04b4a2f31017ad6eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:56.737043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.737843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.741206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1014/2048] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:56 2022] DELETE /identity/v3/projects/9b80d169e3124b04b4a2f31017ad6eb3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:56.745500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:56.755752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1915a4-a24f-4d35-b45a-c5ae19918889 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:56.816150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:56.817217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:56.817568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:56.817969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] PATH_INFO: `/v3/users/3b69f3e0a7f348d29b0290098714190d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:56.819368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b69f3e0a7f348d29b0290098714190d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:56.821360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:56.839100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:56.875291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b69f3e0a7f348d29b0290098714190d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.875966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b69f3e0a7f348d29b0290098714190d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.883345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b69f3e0a7f348d29b0290098714190d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.896078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b69f3e0a7f348d29b0290098714190d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.904494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2a168146-5b00-4e09-bda2-acbd31af0646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b69f3e0a7f348d29b0290098714190d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:56.911336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1035/2049] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:56 2022] DELETE /identity/v3/users/3b69f3e0a7f348d29b0290098714190d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:57.192810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.194200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.218481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c297d5-f9ad-448c-945e-4ead8a84627b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.287950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.288835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.289362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.289362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] PATH_INFO: `/v3/projects/fe123b832379452d89993eb64bc4bbab` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.290594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe123b832379452d89993eb64bc4bbab)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.292538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.324793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:57.342972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe123b832379452d89993eb64bc4bbab', 'request_id': 'req-8060cf2a-4b80-49cd-a539-e62952f71257'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.348695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe123b832379452d89993eb64bc4bbab', 'request_id': 'req-8060cf2a-4b80-49cd-a539-e62952f71257'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.358014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe123b832379452d89993eb64bc4bbab', 'request_id': 'req-8060cf2a-4b80-49cd-a539-e62952f71257'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.369438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] The token cache is being invalidate because project fe123b832379452d89993eb64bc4bbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:57.369723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.370365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8060cf2a-4b80-49cd-a539-e62952f71257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.372083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1015/2050] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:57 2022] DELETE /identity/v3/projects/fe123b832379452d89993eb64bc4bbab => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:57.377492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.377960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.378047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.397315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:57.398535 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.463202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1036/2051] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:57.471279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.471578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.471999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.492669 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:57.493906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.503524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1016/2052] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:57.508242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.518990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc88eb-2771-4284-b546-c31bbca4368f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.523968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.524634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.524936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.525193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.525931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:57.526240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.527979 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.557032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1037/2053] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:41:57.561899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.572056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39512706-566f-468d-b7d6-717f5fa20f72 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.577712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.578339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.578613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.578888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.579632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.581548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.611423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1017/2054] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:57.616892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.630058 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259191-4078-4619-8e5e-e54f39689643 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.702852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.704758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.708801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.709552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.709910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.710293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.711288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.713581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.721952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb67384-66a4-4dec-8cc3-cac1c517c754 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.730014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.731373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.731924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.732273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] PATH_INFO: `/v3/users/e49dde056aa546aab3c48af2992d6006` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.733729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49dde056aa546aab3c48af2992d6006)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.738695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1038/2055] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:57.738845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.740622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.752927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23c42-df36-4ed2-96ed-2ed7d461e41e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.758891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.759597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.759885 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.760558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.762319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.764812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.771468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:57.781048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1039/2056] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:57.783644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.793011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49dde056aa546aab3c48af2992d6006'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.794455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49dde056aa546aab3c48af2992d6006'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.795102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361213f-b105-4e4b-aa6c-c74ab1fbae4d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.800461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.801129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.801407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.801682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.802506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.804385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.805475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49dde056aa546aab3c48af2992d6006'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.814127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49dde056aa546aab3c48af2992d6006'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.819089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1040/2057] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:57.821834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.829360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-0b3fa5a1-f284-4841-8d4c-56192cc47ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49dde056aa546aab3c48af2992d6006'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:57.834346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234b8bd-bc64-4bed-be51-45124ada46eb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:57.835598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1018/2058] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:57 2022] DELETE /identity/v3/users/e49dde056aa546aab3c48af2992d6006 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:41:57.840200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.840525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.840827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.860025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:57.860551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.899817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:57.900881 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.901298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.901675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] PATH_INFO: `/v3/projects/c53e273b7cde400c8e9dc8693c733019/users/8c9c77544056402d92fbae62c178988b/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.918346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53e273b7cde400c8e9dc8693c733019, user_id=8c9c77544056402d92fbae62c178988b, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:57.920309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:57.921796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4edfc-f2e8-43ea-a169-7a696bcbfa4a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:57.931852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1019/2059] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:57.943722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:57.945946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:57.955104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1041/2060] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:57 2022] PUT /identity/v3/projects/c53e273b7cde400c8e9dc8693c733019/users/8c9c77544056402d92fbae62c178988b/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:57.958980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:57.959417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:57.959765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:57.994016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] MFA Rules not processed for user `8c9c77544056402d92fbae62c178988b`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:57.995297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.013541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c8d9f-4d1a-484e-b93e-afa4e3c6e35e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.070046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1042/2061] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:58.074364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.075924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.078818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.078818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.078818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.078818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.144436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e5acf-807b-43f2-b74c-68a384815bc4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.145645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.147184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.149441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.150112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.150394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.150671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.151452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.153638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.158760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1020/2062] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:41:58.159237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:58.182941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1043/2063] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:58.188189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.197953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03359a-307d-4d2b-a070-cb55f085be78 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.258115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.258812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.259256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.259397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.260105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.262085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.271516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:58.285825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1021/2064] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:58.290233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.302027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac874c3-8203-405c-8667-c50f070d3e94 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.307676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.308390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.309482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.309785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.310828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.312501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.323713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1044/2065] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:58.328356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.338693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a382e2-6416-4ce3-a6fd-3be936035412 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.343597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.344501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.344775 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.345053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] PATH_INFO: `/v3/projects/6edccfa1a33249068668fe21ce33ebbb/users/d79851cc91cc45cc909e6e18e45220f6/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.361501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edccfa1a33249068668fe21ce33ebbb, user_id=d79851cc91cc45cc909e6e18e45220f6, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.364363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.384945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1022/2066] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:58 2022] PUT /identity/v3/projects/6edccfa1a33249068668fe21ce33ebbb/users/d79851cc91cc45cc909e6e18e45220f6/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:58.389331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.399430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d3194-205b-473a-8c13-aaaeca5d48a9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.459165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.459879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.460165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.460441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.461185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.462846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.474306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1045/2067] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:58.478619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.489760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516b67-6897-43ce-b8d5-e6beaaf2818e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.494418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.495347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.495649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.495908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] PATH_INFO: `/v3/projects/6edccfa1a33249068668fe21ce33ebbb/users/d79851cc91cc45cc909e6e18e45220f6/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.498004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edccfa1a33249068668fe21ce33ebbb, user_id=d79851cc91cc45cc909e6e18e45220f6, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.501217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36117ec-33c8-44ef-97cd-78c039213fd8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.508170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.509914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.516134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1023/2068] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:58 2022] PUT /identity/v3/projects/6edccfa1a33249068668fe21ce33ebbb/users/d79851cc91cc45cc909e6e18e45220f6/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:58.526443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.526837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155299b-527e-4183-a629-59e49a95a3d9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.526967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.527175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.566335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] MFA Rules not processed for user `d79851cc91cc45cc909e6e18e45220f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:58.568134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.598109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.599065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.599371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.599647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] PATH_INFO: `/v3/projects/8715f0e0ec7f4875a347f9054479f4be` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.600962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8715f0e0ec7f4875a347f9054479f4be)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.602876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.653750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8715f0e0ec7f4875a347f9054479f4be', 'request_id': 'req-2e780b18-fe0a-4182-a570-56e5edac301c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:58.657222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1024/2069] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:58.661985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8715f0e0ec7f4875a347f9054479f4be', 'request_id': 'req-2e780b18-fe0a-4182-a570-56e5edac301c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:58.669590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.669965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8715f0e0ec7f4875a347f9054479f4be', 'request_id': 'req-2e780b18-fe0a-4182-a570-56e5edac301c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:58.671626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.686274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] The token cache is being invalidate because project 8715f0e0ec7f4875a347f9054479f4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:41:58.686577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:58.687469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2e780b18-fe0a-4182-a570-56e5edac301c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:58.689226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1046/2070] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:41:58 2022] DELETE /identity/v3/projects/8715f0e0ec7f4875a347f9054479f4be => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:58.696043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7262-570f-4e1b-a90e-7a1d26e9c41a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.696344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7262-570f-4e1b-a90e-7a1d26e9c41a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.696632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7262-570f-4e1b-a90e-7a1d26e9c41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.706222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:58.719683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bdba7262-570f-4e1b-a90e-7a1d26e9c41a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:58.721635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bdba7262-570f-4e1b-a90e-7a1d26e9c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.740565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909b09-96b3-4580-9e8e-41a795037f11 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.792053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1047/2071] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:58.800879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.801232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.801570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.805130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.805865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.806139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.806428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.808358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.821408 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:58.821819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:58.822625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.833325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1048/2072] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:41:58.837507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.847730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8096f5f-ee56-43e2-85fa-8f4f62e94951 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.852308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.852990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.853278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.853555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.854290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:41:58.854587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.856425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.868314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1049/2073] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:41:58.872796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.883371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085db5b-5285-4a89-835a-dddb70ef5613 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.889090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.889818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.890484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.890484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.891165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.893099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.893357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.895136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.906288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1025/2074] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:58.930265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1050/2075] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:41:58.931501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.943454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8dd5-421e-499e-90d8-ff90ce7444ad service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.007574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.008286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.008566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.008842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.009534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.011493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.035153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1026/2076] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:41:58 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:41:59.037017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.048205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f082b-c74f-4264-907b-3e83bfc1d34f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.055580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.056258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.056539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.056811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.057553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.059348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.070564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1051/2077] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:41:59.074584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.085607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bf144-0ae4-4a79-8b3e-d5824008bfa1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.090483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.091178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.091466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.091744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.092480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.094122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2770745-6ff2-41be-8430-dc11c0840236 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.104586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1027/2078] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:41:59.110283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.123187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beff8b-ddd8-4a55-a608-b97dc1dd7182 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.130417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.134244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.134681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.134984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] PATH_INFO: `/v3/projects/7d34b259256e4c828c79cf00a931460c/users/749278dfe7c54385922a6f9a28ae4759/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.155965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d34b259256e4c828c79cf00a931460c, user_id=749278dfe7c54385922a6f9a28ae4759, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.158912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.176174 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1052/2079] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:41:59 2022] PUT /identity/v3/projects/7d34b259256e4c828c79cf00a931460c/users/749278dfe7c54385922a6f9a28ae4759/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:59.188231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.188666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.188984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.221124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] MFA Rules not processed for user `749278dfe7c54385922a6f9a28ae4759`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:59.222491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:59.300841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1028/2080] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:59.312453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.314375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:59.346512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.348076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:59.367918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:59.377295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a9357-5f50-491d-9049-3d50a1717530 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.382700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.383556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.383854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.384135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.393209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.394656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c706e-b640-4dcc-92f2-dff1195d922d service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.405688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1053/2081] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:41:59.421917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a26e4c-d615-4741-a781-b8ce1dba2b6d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.426565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.427474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.427754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.428045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] PATH_INFO: `/v3/users/8c9c77544056402d92fbae62c178988b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.429221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9c77544056402d92fbae62c178988b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.431175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.482380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9c77544056402d92fbae62c178988b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:59.483041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9c77544056402d92fbae62c178988b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:59.491176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9c77544056402d92fbae62c178988b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:59.498548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9c77544056402d92fbae62c178988b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:59.511382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9c77544056402d92fbae62c178988b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:41:59.518044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1029/2082] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:41:59 2022] DELETE /identity/v3/users/8c9c77544056402d92fbae62c178988b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:41:59.522607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.522913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.523376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.545495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:41:59.546794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:59.573589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:59.624893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1054/2083] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:41:59.635117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:59.636795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:59.699451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c351c-18d9-4751-928b-b63af040d099 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:59.703721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:59.704376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:59.704685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:59.704959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:59.714809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:59.716660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:59.728824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1030/2084] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:00.056997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:00.058467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:00.071236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0650d-3a9e-42a3-bbe8-9af38f67cf5b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:00.094490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:00.097658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.098051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.098430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] PATH_INFO: `/v3/projects/c53e273b7cde400c8e9dc8693c733019` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:00.100165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53e273b7cde400c8e9dc8693c733019)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:00.102300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:00.166771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53e273b7cde400c8e9dc8693c733019', 'request_id': 'req-264ce11c-ece3-4454-9070-292212daf2ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.176773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53e273b7cde400c8e9dc8693c733019', 'request_id': 'req-264ce11c-ece3-4454-9070-292212daf2ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.183862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53e273b7cde400c8e9dc8693c733019', 'request_id': 'req-264ce11c-ece3-4454-9070-292212daf2ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.195075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] The token cache is being invalidate because project c53e273b7cde400c8e9dc8693c733019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:00.195381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.196039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.200608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1055/2085] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:00 2022] DELETE /identity/v3/projects/c53e273b7cde400c8e9dc8693c733019 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:00.202501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:00.216859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712efbe-5048-44e1-bec7-a742494f7623 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:00.261603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:00.290115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:00.291017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.291329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.291607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] PATH_INFO: `/v3/users/d79851cc91cc45cc909e6e18e45220f6` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:00.292851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79851cc91cc45cc909e6e18e45220f6)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:00.295263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:00.359564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79851cc91cc45cc909e6e18e45220f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.360461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79851cc91cc45cc909e6e18e45220f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.371025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79851cc91cc45cc909e6e18e45220f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.378194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79851cc91cc45cc909e6e18e45220f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.391694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-35b1670d-1f93-4320-8f8f-18d6fe67034d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79851cc91cc45cc909e6e18e45220f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.397681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1031/2086] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:00 2022] DELETE /identity/v3/users/d79851cc91cc45cc909e6e18e45220f6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:00.677856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:00.679682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:00.699088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce11c-ece3-4454-9070-292212daf2ed admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:00.758943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:00.759898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.760185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.760465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] PATH_INFO: `/v3/projects/6edccfa1a33249068668fe21ce33ebbb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:00.761814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edccfa1a33249068668fe21ce33ebbb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:00.763991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:00.815682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edccfa1a33249068668fe21ce33ebbb', 'request_id': 'req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.820047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:00.827274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edccfa1a33249068668fe21ce33ebbb', 'request_id': 'req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.833776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edccfa1a33249068668fe21ce33ebbb', 'request_id': 'req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.848217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] The token cache is being invalidate because project 6edccfa1a33249068668fe21ce33ebbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:00.848511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.849175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-afe8a9a9-689f-45a7-a893-f61dbcdfd72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:00.850887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1056/2087] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:00 2022] DELETE /identity/v3/projects/6edccfa1a33249068668fe21ce33ebbb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:00.856391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.856784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.857132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:00.877864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:00.879093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:00.940907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1032/2088] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:00.949101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.949440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.949771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:00.968134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:00.969364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:00.978655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1057/2089] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:00.982473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:00.992702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9764cbf-8c1a-4981-9f75-dd971aeea8c6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:00.998214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:00.998883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:00.999197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:00.999470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.000228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:01.000500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.002108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.018825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1033/2090] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:42:01.019821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.029848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4fec0-713b-429b-bc9d-5a43339d7af7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.035424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.036086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.036367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.036634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.037469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.039453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.066518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1058/2091] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:01.071075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.081630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8d43-2f7e-4316-97de-4660305c2c97 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.131543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:01.155545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.156234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.156518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.156811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.157647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.159614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.183242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1034/2092] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:01.187560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.198091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84553dc5-dff6-40cb-b888-5af89fb22629 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.203092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.203762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.204046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.204324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.205071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.206703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.217432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1059/2093] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:01.221732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.231869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b4862-a842-4660-a63d-80a3922d2736 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.236785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.237447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.237732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.238010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.238737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.240360 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fa37b-5f19-4c53-8652-70a1698a1040 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.250986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1035/2094] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:01.264373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.275620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432ee99-c8e0-43d4-a888-3614ac84db6a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.282016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.282908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.283226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.283472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] PATH_INFO: `/v3/projects/5066f4a96d424a7987705978a9fd22f8/users/e32476f1adb24538aacaf575a7cf2139/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.300284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5066f4a96d424a7987705978a9fd22f8, user_id=e32476f1adb24538aacaf575a7cf2139, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.302945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.321417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1060/2095] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:01 2022] PUT /identity/v3/projects/5066f4a96d424a7987705978a9fd22f8/users/e32476f1adb24538aacaf575a7cf2139/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:01.333346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.333845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.333942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.365733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] MFA Rules not processed for user `e32476f1adb24538aacaf575a7cf2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:01.367167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:01.434596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1036/2096] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:01.441234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.516750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce71bf-a6da-403f-a957-dc34b7b85c52 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.521667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.522336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.522756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.523041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.523761 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.525721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.557898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1061/2097] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:01.562102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.572923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c84b8c-dc58-44fc-840d-c2552cf17295 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.646441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.647280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.647663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.648036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.648816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.650820 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.673871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1037/2098] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:01.677829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.688437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77697c67-28d7-4d26-8ff7-5f2d72a020c1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.693752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.694446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.694699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.694971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.695805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.697411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.709338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1062/2099] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:01.715036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.726033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aeba1-a4e0-4729-9a69-565de3032a6f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.730740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.731854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.732206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.732572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] PATH_INFO: `/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.750441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8457ed45bbc64ff1bc8f79bbfbf81965, user_id=0e55eeaaad6d49ef8e1d7e7fa9442ea5, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.753780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.774338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1038/2100] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:01 2022] PUT /identity/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:01.778483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.789235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae796ff-b6df-433d-894d-ce6310874b9b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.841398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:01.852376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.853093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.853391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.853999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.854899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.856864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f448c6-de3e-4a62-a816-15ca901b7764 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.867462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1063/2101] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:01.872060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:01.882935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b52-5312-4e03-b953-fdb93e60d72f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:01.888114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:01.889037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.889337 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.889598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] PATH_INFO: `/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.891611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8457ed45bbc64ff1bc8f79bbfbf81965, user_id=0e55eeaaad6d49ef8e1d7e7fa9442ea5, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:01.894257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:01.904983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1039/2102] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:01 2022] PUT /identity/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:01.915319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:01.915614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:01.915912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:01.946205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] MFA Rules not processed for user `0e55eeaaad6d49ef8e1d7e7fa9442ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:01.947806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:02.023166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1064/2103] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:02.033969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:02.035742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:02.042369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:02.093662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb697-c20e-4db9-afc7-d16ea22e4dbb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:02.097445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:02.098096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:02.098380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:02.098657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:02.108140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:02.109580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cdaa5-dc8c-4fe4-be3a-e2c140a60154 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:02.118853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1040/2104] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:02.839543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:02.909608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a5835-c6b4-4127-a751-cabf551588d3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:02.914206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:02.915133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:02.915414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:02.915688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] PATH_INFO: `/v3/users/e32476f1adb24538aacaf575a7cf2139` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:02.917031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32476f1adb24538aacaf575a7cf2139)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:02.919640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:02.991023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32476f1adb24538aacaf575a7cf2139'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:02.991734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32476f1adb24538aacaf575a7cf2139'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.004438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32476f1adb24538aacaf575a7cf2139'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.026041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32476f1adb24538aacaf575a7cf2139'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.029273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:03.035848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32476f1adb24538aacaf575a7cf2139'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.043137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1065/2105] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:02 2022] DELETE /identity/v3/users/e32476f1adb24538aacaf575a7cf2139 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:42:03.049239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:03.049766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:03.049858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:03.073770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:03.075191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:03.140719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1041/2106] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:03.150438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:03.151935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:03.212101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72ee1a-c95f-47ce-8527-a6d632162d7b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:03.218575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:03.219533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:03.219977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:03.220417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:03.234370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:03.235842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:03.246502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1066/2107] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:42:03.554968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:03.556549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:03.589825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaa724-064d-4b55-9231-a6cc3e250dc5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:03.595196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:03.596078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:03.596418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:03.596787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] PATH_INFO: `/v3/projects/5066f4a96d424a7987705978a9fd22f8` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:03.597892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5066f4a96d424a7987705978a9fd22f8)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:03.599888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:03.671169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5066f4a96d424a7987705978a9fd22f8', 'request_id': 'req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.679472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5066f4a96d424a7987705978a9fd22f8', 'request_id': 'req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.692220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5066f4a96d424a7987705978a9fd22f8', 'request_id': 'req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.705495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] The token cache is being invalidate because project 5066f4a96d424a7987705978a9fd22f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:03.705787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.706562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.711077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1042/2108] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:03 2022] DELETE /identity/v3/projects/5066f4a96d424a7987705978a9fd22f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:03.712230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:03.723045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f4e76-961e-48b7-ad96-fba038dcb6df service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:03.800632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:03.801527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:03.801951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:03.802234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] PATH_INFO: `/v3/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:03.803480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e55eeaaad6d49ef8e1d7e7fa9442ea5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:03.805571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:03.872589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e55eeaaad6d49ef8e1d7e7fa9442ea5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.873611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e55eeaaad6d49ef8e1d7e7fa9442ea5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.885895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e55eeaaad6d49ef8e1d7e7fa9442ea5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.901704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e55eeaaad6d49ef8e1d7e7fa9442ea5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.913628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b8ff11e8-cc67-4335-adbb-e2d994a6b30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e55eeaaad6d49ef8e1d7e7fa9442ea5'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:03.921021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1067/2109] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:03 2022] DELETE /identity/v3/users/0e55eeaaad6d49ef8e1d7e7fa9442ea5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:04.276165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.277724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:04.293090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddccf3b-bae3-49bc-9a41-d5c095ac1e31 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.327751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:04.360450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.361335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.361632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.361910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] PATH_INFO: `/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.363143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8457ed45bbc64ff1bc8f79bbfbf81965)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.365102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.417754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8457ed45bbc64ff1bc8f79bbfbf81965', 'request_id': 'req-bcfeab89-e664-4f57-9927-470d0ae20b38'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:04.423865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8457ed45bbc64ff1bc8f79bbfbf81965', 'request_id': 'req-bcfeab89-e664-4f57-9927-470d0ae20b38'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:04.433117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8457ed45bbc64ff1bc8f79bbfbf81965', 'request_id': 'req-bcfeab89-e664-4f57-9927-470d0ae20b38'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:04.444467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] The token cache is being invalidate because project 8457ed45bbc64ff1bc8f79bbfbf81965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:04.444755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:04.445418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bcfeab89-e664-4f57-9927-470d0ae20b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:04.447008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1043/2110] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:04 2022] DELETE /identity/v3/projects/8457ed45bbc64ff1bc8f79bbfbf81965 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:04.452609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.452905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.453184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.472808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:04.474045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:04.484870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:04.541383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1068/2111] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:04.550437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.550729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.551013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.570610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:04.571965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:04.581051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1044/2112] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:04.587206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.598364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c212bd-dd61-4b48-b8b3-6d780b44cdae None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.604658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.605342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.605623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.605892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.606639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:04.606944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.608599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.621740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1069/2113] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:42:04.626725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.636833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63863642-e7e7-4a91-a21e-cd435bf774ef None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.641967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.642585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.642866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.643169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.643882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.645768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.673719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1045/2114] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:04 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:04.678030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.689349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750524-a3bc-4ef6-b3fc-d3c73700da7a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.751706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.752380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.752651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.752929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.753631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.755623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.778046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1070/2115] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:04 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:04.782585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.794049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a834-2386-49f2-86d4-9a8a95483057 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.800842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.801624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.801893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.802151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.802914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.804572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.817210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1046/2116] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:04.820488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.831523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d7b5c-051e-42ac-b207-cceb7eb44b7c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.836818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.837513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.837797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.838071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.838804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.840432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6f310-e173-475e-bb74-80e3bb16f0b0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.851729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1071/2117] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:04.856054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:04.866813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825ca1a-6947-4bd1-adfb-609beffee080 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:04.872117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:04.873120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.873321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.873601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] PATH_INFO: `/v3/projects/0d40ada6be40480681cd76e1a6ad9085/users/299eb0ba19a244ae98d4f55d84d260ad/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.890980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d40ada6be40480681cd76e1a6ad9085, user_id=299eb0ba19a244ae98d4f55d84d260ad, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:04.893665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:04.908910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1047/2118] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:04 2022] PUT /identity/v3/projects/0d40ada6be40480681cd76e1a6ad9085/users/299eb0ba19a244ae98d4f55d84d260ad/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:04.918948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:04.919551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:04.919551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:04.950618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] MFA Rules not processed for user `299eb0ba19a244ae98d4f55d84d260ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:04.952052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:05.027248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1072/2119] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:05.028609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.090894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:05.095286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cf38b-9514-4fe5-9daf-93e77255d3d3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.102136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.102822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.103168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.103690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.104514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.107080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.133510 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1048/2120] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:05 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:05.137923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.149563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd82013-b4a6-49f5-98f9-4153288a46c1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.158044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:05.207964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.208638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.208914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.209190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.210537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.211860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.232792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1073/2121] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:05 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:05.236831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.247596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91471c9e-4bff-4d3e-893f-4de6cb60a4b8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.254355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.255187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.255550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.255878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.257244 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.258917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.270657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1049/2122] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:05.274748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.285457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf04b-d283-439b-92a0-2ad79fbb5c8b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.290991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.291890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.292186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.292452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] PATH_INFO: `/v3/projects/08e67be2ba3e4813be49baa148d627f1/users/9dde4dce08514c69b631027e8dac7100/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.312578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e67be2ba3e4813be49baa148d627f1, user_id=9dde4dce08514c69b631027e8dac7100, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.315602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.331807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1074/2123] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:05 2022] PUT /identity/v3/projects/08e67be2ba3e4813be49baa148d627f1/users/9dde4dce08514c69b631027e8dac7100/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:05.336103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.347581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f942d10-b7c0-4eec-a6ac-b2a352c00de1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.408054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.408722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.409005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.409274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.410025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.411652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c67e2e-7a84-4d2e-89b2-608f75eb26b0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.425025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1050/2124] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:05.426222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.436993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048314-17f1-4e69-bbeb-b91190313219 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.441711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.442584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.442863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.443160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] PATH_INFO: `/v3/projects/08e67be2ba3e4813be49baa148d627f1/users/9dde4dce08514c69b631027e8dac7100/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.445234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e67be2ba3e4813be49baa148d627f1, user_id=9dde4dce08514c69b631027e8dac7100, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.447907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.461001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1075/2125] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:05 2022] PUT /identity/v3/projects/08e67be2ba3e4813be49baa148d627f1/users/9dde4dce08514c69b631027e8dac7100/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:05.471336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.471633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.471914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.500409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] MFA Rules not processed for user `9dde4dce08514c69b631027e8dac7100`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:05.501640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:05.574079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1051/2126] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:05.585152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:05.587296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:05.654315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df54b85-e71b-4448-b7d5-d1a21ba6ba88 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:05.659064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:05.659728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:05.659998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:05.660270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:05.670254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:05.671699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ac3fe-88a1-44d8-8044-961d2ad08e4f service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:05.682234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1076/2127] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:06.073123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:06.126935 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:06.139971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76171d3-2450-4aba-9880-e4e9e1e318f8 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:06.146509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:06.147723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:06.148007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:06.148258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] PATH_INFO: `/v3/users/299eb0ba19a244ae98d4f55d84d260ad` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:06.149502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299eb0ba19a244ae98d4f55d84d260ad)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:06.151509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:06.212039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299eb0ba19a244ae98d4f55d84d260ad'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:06.212708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299eb0ba19a244ae98d4f55d84d260ad'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:06.224543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299eb0ba19a244ae98d4f55d84d260ad'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:06.234206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299eb0ba19a244ae98d4f55d84d260ad'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:06.249076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299eb0ba19a244ae98d4f55d84d260ad'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:06.263024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1052/2128] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:06 2022] DELETE /identity/v3/users/299eb0ba19a244ae98d4f55d84d260ad => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:06.265438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:06.266568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:06.267086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:06.308488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:06.310325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:06.383562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1077/2129] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:06.394049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:06.395556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:06.482218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ee294-ad46-43eb-8156-ebd0dd87c0b1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:06.488362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:06.489124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:06.489319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:06.489895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:06.505468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:06.508794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:06.527781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1053/2130] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:06.908088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:06.909406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:06.947809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ad68d-1e22-44d9-9066-f09f6b0044a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:06.952616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:06.953482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:06.953768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:06.954042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] PATH_INFO: `/v3/projects/0d40ada6be40480681cd76e1a6ad9085` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:06.955261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d40ada6be40480681cd76e1a6ad9085)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:06.957181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:06.959467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:07.017560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d40ada6be40480681cd76e1a6ad9085', 'request_id': 'req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.026355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d40ada6be40480681cd76e1a6ad9085', 'request_id': 'req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.034339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d40ada6be40480681cd76e1a6ad9085', 'request_id': 'req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.048736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] The token cache is being invalidate because project 0d40ada6be40480681cd76e1a6ad9085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:07.049141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.049748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.051627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1078/2131] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:06 2022] DELETE /identity/v3/projects/0d40ada6be40480681cd76e1a6ad9085 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:07.055527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:07.065889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc90c-d9f8-4baf-87e6-7f474935dbc8 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:07.134350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:07.135293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:07.135533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:07.135805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] PATH_INFO: `/v3/users/9dde4dce08514c69b631027e8dac7100` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:07.137055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dde4dce08514c69b631027e8dac7100)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:07.139777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:07.165531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:07.206439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dde4dce08514c69b631027e8dac7100'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.207157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dde4dce08514c69b631027e8dac7100'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.216015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dde4dce08514c69b631027e8dac7100'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.223422 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dde4dce08514c69b631027e8dac7100'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.239326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-69fa887b-cc95-4343-bd94-9ecb6b2e3a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dde4dce08514c69b631027e8dac7100'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.246825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1054/2132] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:07 2022] DELETE /identity/v3/users/9dde4dce08514c69b631027e8dac7100 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:07.656980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:07.658573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:07.686395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda94c1-678a-4858-b679-d0c0c0fc3a9a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:07.750864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:07.751916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:07.752871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:07.753155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] PATH_INFO: `/v3/projects/08e67be2ba3e4813be49baa148d627f1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:07.754469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e67be2ba3e4813be49baa148d627f1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:07.756742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:07.814302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e67be2ba3e4813be49baa148d627f1', 'request_id': 'req-6fc3d2ff-0ec2-4299-8756-64789591fb01'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.825932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e67be2ba3e4813be49baa148d627f1', 'request_id': 'req-6fc3d2ff-0ec2-4299-8756-64789591fb01'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.833354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e67be2ba3e4813be49baa148d627f1', 'request_id': 'req-6fc3d2ff-0ec2-4299-8756-64789591fb01'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.845371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] The token cache is being invalidate because project 08e67be2ba3e4813be49baa148d627f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:07.845571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.846872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6fc3d2ff-0ec2-4299-8756-64789591fb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:07.848670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1079/2133] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:07 2022] DELETE /identity/v3/projects/08e67be2ba3e4813be49baa148d627f1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:07.853329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:07.853792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:07.853911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:07.874311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:07.875807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:07.940184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1055/2134] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:07.948638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:07.948850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:07.949106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:07.974565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:07.975867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:07.985883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1080/2135] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:07.990493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.000761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9414d-fa9f-48ee-b96b-c479c0068dff None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.005631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.006612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.007086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.007364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.008130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:08.008537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.010142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.027828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1056/2136] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:08.032376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.042162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997a187-3054-4392-a32d-95d88abbe430 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.048017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.048684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.048970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.049238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.049942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.051923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.068028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:08.092639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1081/2137] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:08 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:08.093751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.105189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee59f-4a11-4a5e-9f29-8c5d5e553ee6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.178286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.178993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.179334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.179813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.180860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.182887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.207773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1057/2138] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:08 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:08.212038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.223575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbfa67-8c01-4942-b271-5a54eb1ff125 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.239184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.240250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.240713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.241162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.242093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.243966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.258354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1082/2139] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:08.265856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.279874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d66b5-b71c-4b38-8510-2afebd0f5983 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.295888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.296596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.296882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.297193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.302869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.304708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f998af-c142-4115-9527-0732adb7d62c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.312806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:08.327422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.327891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1058/2140] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:08.338800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d476cd-9bfe-4c43-a4ef-0b12a15ade5c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.348796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.350092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.350450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.350800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] PATH_INFO: `/v3/projects/40886313a5bd43ed8cc0c3eb33e4379e/users/f942e1ee372443a9a7acff3089144f30/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.377892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=40886313a5bd43ed8cc0c3eb33e4379e, user_id=f942e1ee372443a9a7acff3089144f30, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.384021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.404934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1083/2141] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:08 2022] PUT /identity/v3/projects/40886313a5bd43ed8cc0c3eb33e4379e/users/f942e1ee372443a9a7acff3089144f30/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:08.415392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.415745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.415967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.458609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] MFA Rules not processed for user `f942e1ee372443a9a7acff3089144f30`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:08.460322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:08.567665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1059/2142] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:08.579232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.581056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:08.654268 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15fe9f-70aa-4fc4-9c5a-9292545b30aa admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.658208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.659043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.659412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.659693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.670480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.671957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f48a5-00dd-47e6-8e7d-7d9f7898075c service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.681791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1084/2143] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:08.700879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.776399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8d72a-1c0b-42b1-8f40-4077c41222eb None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:08.781433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:08.782448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.782794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.783102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] PATH_INFO: `/v3/users/f942e1ee372443a9a7acff3089144f30` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.784313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f942e1ee372443a9a7acff3089144f30)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:08.786261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:08.835863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f942e1ee372443a9a7acff3089144f30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:08.836531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f942e1ee372443a9a7acff3089144f30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:08.845468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f942e1ee372443a9a7acff3089144f30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:08.852584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f942e1ee372443a9a7acff3089144f30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:08.864950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f942e1ee372443a9a7acff3089144f30'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:08.871025 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1060/2144] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:08 2022] DELETE /identity/v3/users/f942e1ee372443a9a7acff3089144f30 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:08.876187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186a15-37be-4d70-ae0a-ec54519d6c1e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:08.876486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186a15-37be-4d70-ae0a-ec54519d6c1e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:08.876953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186a15-37be-4d70-ae0a-ec54519d6c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:08.894880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-36186a15-37be-4d70-ae0a-ec54519d6c1e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:08.896327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-36186a15-37be-4d70-ae0a-ec54519d6c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:08.958295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1085/2145] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:08.968480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:08.969985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:08.982371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:09.063898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8429b53-5114-4670-8604-b1f2fb2d583c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:09.068635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:09.069328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.069622 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.069916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.080617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:09.082184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493616f-0e7e-45fd-8613-203b030bdc39 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:09.092453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1061/2146] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:42:09.255845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.257002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.258065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.273176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1086/2147] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:42:09 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:42:09.302968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.303582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.303922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.324019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:09.326511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:09.426290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1062/2148] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:09.446312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:09.448940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:09.478173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd94d29-9b0a-4890-9dc7-5cb232f9243d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:09.491424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:09.492171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.492465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.492743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.504905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:09.507234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaae6c1-6b5f-4973-b9fd-f752b51b6b35 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:09.522964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1087/2149] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:09.654207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:09.660613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:09.690438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73686e-0f83-469e-b955-d43c09aa2097 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:09.715270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:09.717978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.718279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.718556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] PATH_INFO: `/v3/projects/40886313a5bd43ed8cc0c3eb33e4379e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.722790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40886313a5bd43ed8cc0c3eb33e4379e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:09.725083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:09.784863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40886313a5bd43ed8cc0c3eb33e4379e', 'request_id': 'req-5041b51a-271e-47d9-8f4d-81548894626c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:09.795708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40886313a5bd43ed8cc0c3eb33e4379e', 'request_id': 'req-5041b51a-271e-47d9-8f4d-81548894626c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:09.805587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40886313a5bd43ed8cc0c3eb33e4379e', 'request_id': 'req-5041b51a-271e-47d9-8f4d-81548894626c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:09.819909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] The token cache is being invalidate because project 40886313a5bd43ed8cc0c3eb33e4379e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:09.820220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:09.820978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5041b51a-271e-47d9-8f4d-81548894626c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:09.826270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1063/2150] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:09 2022] DELETE /identity/v3/projects/40886313a5bd43ed8cc0c3eb33e4379e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:09.828012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.828811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.829256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.848771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:09.850383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:09.902488 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:09.924407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1088/2151] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:09.933096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:09.933967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:09.934275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:09.972140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:09.973289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:09.983678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1064/2152] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:09.988332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:09.999301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879882f-6fd2-4720-ba44-61aa240c4b96 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.004597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.005382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.005735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.006080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.007258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:10.007635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.009348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.027357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1089/2153] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:10.028887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.038611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16215aa-1c06-4347-8128-2bc756221b65 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.043222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.044093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.044416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.044699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.045403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.047329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.079178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1065/2154] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:10 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:10.080439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.092207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c15ed-df30-4917-8865-668193945923 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.154618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.155336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.155616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.155888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.156590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.158531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.184999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1090/2155] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:10 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:10.187039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.193111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:10.198744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7adaf-44d1-4d01-881c-5c308d93b407 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.204815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.205643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.205929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.206229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.206968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.209222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.220155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1066/2156] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:10.224266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.234712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889de4aa-eb6c-4807-b92b-82555ba3e1a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.239854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.240552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.240853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.241129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.241859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.243509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a9a5c-84b1-4327-a686-d7c21db6c36f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.254803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1091/2157] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:42:10.260059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.271617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50b6c-74e1-454a-8e9a-785d8cd36422 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.276136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.277057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.277336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.277608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] PATH_INFO: `/v3/projects/8bd9fdce43fb4510ba286e33a8450246/users/1de6dcc2fd8d470f8e3bdac941e62920/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.296112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd9fdce43fb4510ba286e33a8450246, user_id=1de6dcc2fd8d470f8e3bdac941e62920, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.299666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.329438 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1067/2158] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:10 2022] PUT /identity/v3/projects/8bd9fdce43fb4510ba286e33a8450246/users/1de6dcc2fd8d470f8e3bdac941e62920/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:10.332864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.333154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.333447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.377939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] MFA Rules not processed for user `1de6dcc2fd8d470f8e3bdac941e62920`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:10.379932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:10.472533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1092/2159] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:10.477107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.589960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50436-b569-4cb3-b0e0-58474e28167e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.594754 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.595468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.595749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.596021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.596723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.598623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.627249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1068/2160] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:10 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:10.628463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.639878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b7cb8-ddc4-4783-9d66-8da9f48bcdb7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.679739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:10.697625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.698331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.698644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.698930 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.699731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.702063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.724172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1093/2161] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:10 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:10.728462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.739227 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c9c9b-f705-481c-8366-064d8846c7a0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.744095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.744766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.745047 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.745328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.746091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.748027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.760382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1069/2162] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:10.764599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.775565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c852f8-797c-4efb-9505-6a367d84ed7a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.781091 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.782023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.782309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.782584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] PATH_INFO: `/v3/projects/1a93535e57474e90b85be9a7eae1c28a/users/71024959aea64cc297696db0c8ce546f/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.808320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a93535e57474e90b85be9a7eae1c28a, user_id=71024959aea64cc297696db0c8ce546f, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.811575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.812492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.813819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:10.841596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1094/2163] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:10 2022] PUT /identity/v3/projects/1a93535e57474e90b85be9a7eae1c28a/users/71024959aea64cc297696db0c8ce546f/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:10.841596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.864579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b89ca-f3f1-4b9f-9b1d-8be8759f2d22 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.946602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a755cd9-f722-4b4c-ab0c-51ffcb055636 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.954454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.955371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.955762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.956145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.957039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.958785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.972379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1095/2164] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:10.976439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:10.979760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.980977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.981634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.982035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] PATH_INFO: `/v3/users/f0a202a9fc384d818f702763e1f5c0c1` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:10.985906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a202a9fc384d818f702763e1f5c0c1)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:10.987944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:10.989925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c271e7c-8e22-4989-b405-509ec1176535 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:10.996782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:10.998005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:10.998308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:10.999307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] PATH_INFO: `/v3/projects/1a93535e57474e90b85be9a7eae1c28a/users/71024959aea64cc297696db0c8ce546f/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:11.002184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a93535e57474e90b85be9a7eae1c28a, user_id=71024959aea64cc297696db0c8ce546f, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:11.007308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd575dc4-f3d8-43b1-ab16-87372603af92 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:11.038227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1096/2165] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:10 2022] PUT /identity/v3/projects/1a93535e57474e90b85be9a7eae1c28a/users/71024959aea64cc297696db0c8ce546f/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:11.040379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:11.040745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:11.041098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:11.047279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:11.080864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] MFA Rules not processed for user `71024959aea64cc297696db0c8ce546f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:11.081574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:11.084605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a202a9fc384d818f702763e1f5c0c1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.085905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a202a9fc384d818f702763e1f5c0c1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.128176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a202a9fc384d818f702763e1f5c0c1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.149352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a202a9fc384d818f702763e1f5c0c1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.183386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a202a9fc384d818f702763e1f5c0c1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.195758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1070/2166] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:10 2022] DELETE /identity/v3/users/f0a202a9fc384d818f702763e1f5c0c1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:11.204804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1097/2167] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:11.218814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:11.220864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:11.331780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb1c5d-0253-4444-a36f-b71574c95aec admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:11.337630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:11.338534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:11.338848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:11.339170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:11.341363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:11.468862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:11.501808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:11.504187 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:11.513841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:11.517913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1071/2168] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:11.590969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc1bbd-5ad9-4cc2-9440-0c293181eb0c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:11.666613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:11.668028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:11.668647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:11.669171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] PATH_INFO: `/v3/projects/59a2245edc014b7082696f35af04ba26` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:11.671661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a2245edc014b7082696f35af04ba26)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:11.675109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:11.755819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a2245edc014b7082696f35af04ba26', 'request_id': 'req-d619c2ea-3f78-483c-92b4-db44bf2beefc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.768168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a2245edc014b7082696f35af04ba26', 'request_id': 'req-d619c2ea-3f78-483c-92b4-db44bf2beefc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.777738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a2245edc014b7082696f35af04ba26', 'request_id': 'req-d619c2ea-3f78-483c-92b4-db44bf2beefc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.796405 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] The token cache is being invalidate because project 59a2245edc014b7082696f35af04ba26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:11.798293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.799596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:11.802015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1098/2169] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:11 2022] DELETE /identity/v3/projects/59a2245edc014b7082696f35af04ba26 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:42:11.806506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:11.808762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:11.829001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436d29-ee7d-414f-9c5d-e1c50892139b service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:11.953779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:11.955836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:11.956174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:11.956511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] PATH_INFO: `/v3/users/0d338967a4f2409d885a6330a68c6005` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:11.959152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d338967a4f2409d885a6330a68c6005)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:11.963629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.008567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:12.075001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d338967a4f2409d885a6330a68c6005'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.077104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d338967a4f2409d885a6330a68c6005'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.093258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d338967a4f2409d885a6330a68c6005'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.111136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d338967a4f2409d885a6330a68c6005'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.145910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d338967a4f2409d885a6330a68c6005'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.154262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1072/2170] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:11 2022] DELETE /identity/v3/users/0d338967a4f2409d885a6330a68c6005 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:12.277106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.278931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.296354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619c2ea-3f78-483c-92b4-db44bf2beefc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:12.350258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:12.361324 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:12.362481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.363012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.364111 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] PATH_INFO: `/v3/users/749278dfe7c54385922a6f9a28ae4759` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.366119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749278dfe7c54385922a6f9a28ae4759)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:12.368690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.409076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.410507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.427816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b11814-555f-4a0c-8614-fc942adb0927 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:12.436250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:12.438987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.439316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.439596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] PATH_INFO: `/v3/projects/79d50f4bd85a452e8c46056da9a06510` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.441969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d50f4bd85a452e8c46056da9a06510)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:12.443985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.460038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749278dfe7c54385922a6f9a28ae4759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.460828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749278dfe7c54385922a6f9a28ae4759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.469278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749278dfe7c54385922a6f9a28ae4759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.489299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749278dfe7c54385922a6f9a28ae4759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.504755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-14656550-92f2-4225-9b37-b28f66058663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749278dfe7c54385922a6f9a28ae4759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.512239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1099/2171] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:12 2022] DELETE /identity/v3/users/749278dfe7c54385922a6f9a28ae4759 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:12.516592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.516893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.517516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.524559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d50f4bd85a452e8c46056da9a06510', 'request_id': 'req-618fd23b-4780-46e9-b204-bf383f29cb3b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.541391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d50f4bd85a452e8c46056da9a06510', 'request_id': 'req-618fd23b-4780-46e9-b204-bf383f29cb3b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.544229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:12.550497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.557116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d50f4bd85a452e8c46056da9a06510', 'request_id': 'req-618fd23b-4780-46e9-b204-bf383f29cb3b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.578922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] The token cache is being invalidate because project 79d50f4bd85a452e8c46056da9a06510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:12.579269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.580296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-618fd23b-4780-46e9-b204-bf383f29cb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:12.582946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1073/2172] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:12 2022] DELETE /identity/v3/projects/79d50f4bd85a452e8c46056da9a06510 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:42:12.589812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53819d70-9f7c-4bd4-a081-cf7340f72813 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.590239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53819d70-9f7c-4bd4-a081-cf7340f72813 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.590586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53819d70-9f7c-4bd4-a081-cf7340f72813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.624708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-53819d70-9f7c-4bd4-a081-cf7340f72813 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:12.626498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-53819d70-9f7c-4bd4-a081-cf7340f72813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.644691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1100/2173] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:12.654779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.656442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.718957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1074/2174] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:12.720944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.721232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.721509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.751747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d47f80-11a0-48f8-93fe-633fc9a3b156 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:12.753033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:12.754381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:12.757116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:12.757958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.758389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.758827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.767089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1075/2175] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:42:12.771486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.773812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:12.775487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.784050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8d5d1-3f03-4b35-afba-d4d1a2186419 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:12.787828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1101/2176] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:42:12.790077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:12.790846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.791236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.791651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.792437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:12.792918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:12.794809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.811628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:12.830008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1076/2177] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:42:12.837149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.850308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad5702-2879-4b65-b1da-e2ce74015915 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:12.857136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:12.858163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:12.858546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:12.859189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:12.859771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:12.861825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:12.880312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:12.912049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1102/2178] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:12 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:12.912897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:12.938087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769c344-b302-4c4e-8b27-0fb1e36da1bd admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.088934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.089805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.090123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.090427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.091328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.095138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a042f9-29dc-4022-bd45-c529938ff526 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.155566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1077/2179] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:12 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:13.156443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.172265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7321-59f3-4f93-8e10-98c9a28c7d45 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.193827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.194645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.195171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.195522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.196388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.210271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.210576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.210966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.211443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.222823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1103/2180] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:13.231145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.254973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] MFA Rules not processed for user `f9aced17fa4848ecb0d5be31b87e1e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:13.256628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683c1f3-116b-4d18-9628-9ad58bdc6dac admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.257677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.270833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.272029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.273049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.273538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.276013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.284483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.304134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1104/2181] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:13.306760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.311139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.333878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c42b7-ce5f-490e-8e0d-b040a3bc3ceb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.354760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.355695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.355980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.356305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] PATH_INFO: `/v3/projects/7d34b259256e4c828c79cf00a931460c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.357803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d34b259256e4c828c79cf00a931460c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.359786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.379850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1078/2182] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:42:13.380959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.395877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b524a89-7f1a-4a0d-a395-b4556e5668a4 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.405462 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.406569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.406854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.407315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] PATH_INFO: `/v3/projects/b8f18a3b6e6746108a836d285a840bc7/users/25d491b43061435a8fb61fd450ff9e6b/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.427036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f18a3b6e6746108a836d285a840bc7, user_id=25d491b43061435a8fb61fd450ff9e6b, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.430519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ddf1d-419b-4c6a-af0a-c6ce2ff79edf admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.430850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d34b259256e4c828c79cf00a931460c', 'request_id': 'req-86dc575b-a59b-4b0e-9012-d341b6505431'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.447627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d34b259256e4c828c79cf00a931460c', 'request_id': 'req-86dc575b-a59b-4b0e-9012-d341b6505431'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.448046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1079/2183] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:13 2022] PUT /identity/v3/projects/b8f18a3b6e6746108a836d285a840bc7/users/25d491b43061435a8fb61fd450ff9e6b/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:13.451061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168f848-feda-4c30-9e3a-6965c74a082c None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.451548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168f848-feda-4c30-9e3a-6965c74a082c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.452006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168f848-feda-4c30-9e3a-6965c74a082c None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.453523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1080/2184] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:42:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:42:13.458068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.458530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.458812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.460688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d34b259256e4c828c79cf00a931460c', 'request_id': 'req-86dc575b-a59b-4b0e-9012-d341b6505431'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.477978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] The token cache is being invalidate because project 7d34b259256e4c828c79cf00a931460c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:13.478295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.479354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-86dc575b-a59b-4b0e-9012-d341b6505431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.480993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1105/2185] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:13 2022] DELETE /identity/v3/projects/7d34b259256e4c828c79cf00a931460c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:13.490653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.490653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.490653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.502025 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] MFA Rules not processed for user `9b84d7ac4698432e98a30fd39de251f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:13.503434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.518735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] MFA Rules not processed for user `25d491b43061435a8fb61fd450ff9e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:13.520409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.598977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1106/2186] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:13.600056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.601779 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.643042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:13.654291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1081/2187] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:13.655246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.656702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.677993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78395ce3-7733-48b6-a6b9-f1b300947b2d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.684198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.684918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.685204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.685478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.696027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.697653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.706256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:13.711515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1082/2188] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:13.712347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:13.714413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:13.721661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f241a84-b9b2-4c92-8cd3-58bd24f0c398 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.732252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.733400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.733807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.734159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] PATH_INFO: `/v3/users/5ab5b939a13e43469b3a4a48da559c83` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.735762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab5b939a13e43469b3a4a48da559c83)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.738096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.785848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daf1ce-5e21-452d-a71f-4b1e58bc1d93 service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:13.789549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:13.790205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:13.790492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:13.790772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:13.795085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab5b939a13e43469b3a4a48da559c83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.795706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab5b939a13e43469b3a4a48da559c83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.800162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:13.801614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:13.805129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab5b939a13e43469b3a4a48da559c83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.811833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1083/2189] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:13.820865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab5b939a13e43469b3a4a48da559c83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.829261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13315c05-3320-4a17-b1f9-a60762fa1b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab5b939a13e43469b3a4a48da559c83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:13.839984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1107/2190] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:13 2022] DELETE /identity/v3/users/5ab5b939a13e43469b3a4a48da559c83 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:14.034513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.048467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b24f83-f8cd-40fd-a9d6-613d727276f0 service glance] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.148325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.148325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.148325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.148325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] PATH_INFO: `/v3/projects/0a7a93f8b61c48759abc98e8b86ad4e8` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.149169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7a93f8b61c48759abc98e8b86ad4e8)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.149169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.202699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7a93f8b61c48759abc98e8b86ad4e8', 'request_id': 'req-02806a02-da75-4a2e-8485-dc8bb0eb1475'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:14.211508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7a93f8b61c48759abc98e8b86ad4e8', 'request_id': 'req-02806a02-da75-4a2e-8485-dc8bb0eb1475'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:14.220929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7a93f8b61c48759abc98e8b86ad4e8', 'request_id': 'req-02806a02-da75-4a2e-8485-dc8bb0eb1475'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:14.240007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] The token cache is being invalidate because project 0a7a93f8b61c48759abc98e8b86ad4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:14.240480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:14.241731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02806a02-da75-4a2e-8485-dc8bb0eb1475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:14.247159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1084/2191] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:14 2022] DELETE /identity/v3/projects/0a7a93f8b61c48759abc98e8b86ad4e8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:14.250652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.251162 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.251365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.287548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:14.289075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:14.379886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1108/2192] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:14.389087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.389641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.391077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.423195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:14.424466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:14.435588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1085/2193] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:14.445265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.458490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e01d5-fc86-431c-b440-8da6ef82571d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.467397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.468591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.468937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.469274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.470136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:14.474073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.474073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.493420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1109/2194] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:14.498084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.513553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52f6ed-32d4-48b5-9b40-c2d7cb5e6583 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.519720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.520423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.521520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.521808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.522660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.524683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.544604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:14.570875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1086/2195] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:14 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:14.575635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.587032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a6f74-a3ea-4dbc-ad2e-eb0185ead0af admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.666877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:14.682592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.683507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.683894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.684303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.685048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.687260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.712955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1110/2196] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:14 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:14.714647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.728311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46898-ae60-4593-8bee-8776d7b191ea admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.733233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.733919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.734209 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.734487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.735262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.737060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.748493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1087/2197] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:42:14.754002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.766040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ab637-a8ad-4eab-b96f-a8dd7e19c3e1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.771321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.771990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.772278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.772553 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.773290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.775075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf57f09-a430-413c-9f68-53c6e2620031 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.785860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1111/2198] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:14.790358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.810699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c0ebc-0e14-440e-b76d-93aeee0a25a0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:14.817559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:14.818541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.818896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.819294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] PATH_INFO: `/v3/projects/0907423ad5f14bf9a0680c9cdb78c995/users/86f1f2722c5c4da59fd83036ce1700e1/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.835718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0907423ad5f14bf9a0680c9cdb78c995, user_id=86f1f2722c5c4da59fd83036ce1700e1, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:14.838390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:14.854640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1088/2199] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:14 2022] PUT /identity/v3/projects/0907423ad5f14bf9a0680c9cdb78c995/users/86f1f2722c5c4da59fd83036ce1700e1/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:14.860884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:14.861216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:14.861593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:14.883644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:14.884856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:14.949189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1112/2200] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:14.959731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:14.961721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:15.023529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47228a-edbd-418e-b0c0-96ad651233a8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:15.027126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:15.027780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:15.028137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:15.028427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:15.038598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:15.040067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e695-efb6-42ff-a87f-745a39ce0c27 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:15.050534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1089/2201] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:42:15.562164 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:15.563750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:15.575967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc96f3-d2d6-4365-b03b-0717ccfa32bb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:15.580804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:15.581682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:15.581949 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:15.582222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] PATH_INFO: `/v3/users/1de6dcc2fd8d470f8e3bdac941e62920` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:15.583433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de6dcc2fd8d470f8e3bdac941e62920)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:15.585364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:15.637616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de6dcc2fd8d470f8e3bdac941e62920'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:15.638389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de6dcc2fd8d470f8e3bdac941e62920'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:15.646037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de6dcc2fd8d470f8e3bdac941e62920'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:15.659886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de6dcc2fd8d470f8e3bdac941e62920'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:15.670662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de6dcc2fd8d470f8e3bdac941e62920'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:15.677374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1113/2202] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:15 2022] DELETE /identity/v3/users/1de6dcc2fd8d470f8e3bdac941e62920 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:15.682086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:15.683157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:15.683628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:15.717721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:15.719589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:15.799094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1090/2203] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:15.810198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:15.813630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:15.898130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:15.902928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4cf3-5229-4920-b268-7284116ac05d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:15.909499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:15.910691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:15.911322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:15.911867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:15.924685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:15.926353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:15.938319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1114/2204] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:42:16.677633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:16.679305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:16.704558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2a74f-22a9-4825-b32f-29744cdd6f13 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:16.714894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:16.716098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:16.716384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:16.716661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] PATH_INFO: `/v3/projects/8bd9fdce43fb4510ba286e33a8450246` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:16.717882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd9fdce43fb4510ba286e33a8450246)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:16.719948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:16.833626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd9fdce43fb4510ba286e33a8450246', 'request_id': 'req-c60f6985-fbd9-48aa-9576-74e14b4e13c2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:16.842191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:16.857315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd9fdce43fb4510ba286e33a8450246', 'request_id': 'req-c60f6985-fbd9-48aa-9576-74e14b4e13c2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:16.875443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd9fdce43fb4510ba286e33a8450246', 'request_id': 'req-c60f6985-fbd9-48aa-9576-74e14b4e13c2'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:16.892325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] The token cache is being invalidate because project 8bd9fdce43fb4510ba286e33a8450246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:16.892325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:16.893210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:16.898014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1091/2205] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:16 2022] DELETE /identity/v3/projects/8bd9fdce43fb4510ba286e33a8450246 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:16.899198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:16.914398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2424f-3bdc-407a-b346-e8d6f273602a service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:16.993391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:16.995738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:16.995738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:16.995738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] PATH_INFO: `/v3/users/71024959aea64cc297696db0c8ce546f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:16.996505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71024959aea64cc297696db0c8ce546f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:16.998681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:17.055625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71024959aea64cc297696db0c8ce546f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.056713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71024959aea64cc297696db0c8ce546f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.067001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71024959aea64cc297696db0c8ce546f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.081490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71024959aea64cc297696db0c8ce546f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.093444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-60d298bc-ce94-41fb-b2fd-6f1887874d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71024959aea64cc297696db0c8ce546f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.102443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1115/2206] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:16 2022] DELETE /identity/v3/users/71024959aea64cc297696db0c8ce546f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:17.446720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:17.448379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:17.465524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f6985-fbd9-48aa-9576-74e14b4e13c2 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:17.608631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:17.609537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:17.609832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:17.610140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] PATH_INFO: `/v3/projects/1a93535e57474e90b85be9a7eae1c28a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:17.611533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a93535e57474e90b85be9a7eae1c28a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:17.613504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:17.687015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a93535e57474e90b85be9a7eae1c28a', 'request_id': 'req-b00b4da1-448b-4f27-8567-caa207dc96c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.693309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a93535e57474e90b85be9a7eae1c28a', 'request_id': 'req-b00b4da1-448b-4f27-8567-caa207dc96c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.704783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a93535e57474e90b85be9a7eae1c28a', 'request_id': 'req-b00b4da1-448b-4f27-8567-caa207dc96c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.717794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] The token cache is being invalidate because project 1a93535e57474e90b85be9a7eae1c28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:17.718080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.718774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b00b4da1-448b-4f27-8567-caa207dc96c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:17.720374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1092/2207] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:17 2022] DELETE /identity/v3/projects/1a93535e57474e90b85be9a7eae1c28a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:17.725465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:17.725863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:17.726409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:17.745402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:17.746845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:17.840015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1116/2208] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:17.849022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:17.849421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:17.849771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:17.872370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:17.874004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:17.883099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1093/2209] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:17.887579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:17.898248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f01f6-fbb3-49ed-b03e-0f124c64f892 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:17.903850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:17.904634 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:17.904919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:17.905195 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:17.905941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:17.906876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:17.907927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:17.915291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:17.921150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1117/2210] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:17.926116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:17.945646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5b425-3437-414e-82e3-59bdfe93958a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:17.955170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:17.955903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:17.956188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:17.956512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:17.957231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:17.959301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:17.993677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1094/2211] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:17 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:17.998555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:18.009133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf3c75-4a13-491c-a1cb-37bb86bd752c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:18.096995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:18.098068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.098361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.099079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.099380 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:18.101317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:18.125503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1118/2212] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:17 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:18.129505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:18.141102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2a1b-0a65-49a9-a065-722a8d81720e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:18.147637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:18.148425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.148842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.149169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.149875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:18.151563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:18.163232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1095/2213] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:18.167587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:18.179031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a680033-0b74-4ede-880c-5ca546b47ffe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:18.183811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:18.184836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.185129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.185426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.186492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:18.188431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90940ed-a4c2-41f2-a0f9-7bf9651da112 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:18.201435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1119/2214] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:18.205674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:18.217139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ac43-4c82-4adf-bee8-f74aac393278 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:18.222721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:18.223662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.223947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.224226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] PATH_INFO: `/v3/projects/df15611c4bb346108d70f6a0c582ab84/users/b39c11b84e144804afc26d0ef9703022/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.240992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df15611c4bb346108d70f6a0c582ab84, user_id=b39c11b84e144804afc26d0ef9703022, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:18.244425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:18.263208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1096/2215] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:18 2022] PUT /identity/v3/projects/df15611c4bb346108d70f6a0c582ab84/users/b39c11b84e144804afc26d0ef9703022/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:42:18.273863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fd994-2717-4690-b029-1243c1729527 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.273863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fd994-2717-4690-b029-1243c1729527 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.273863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fd994-2717-4690-b029-1243c1729527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.303038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-345fd994-2717-4690-b029-1243c1729527 None None] MFA Rules not processed for user `b39c11b84e144804afc26d0ef9703022`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:18.304346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-345fd994-2717-4690-b029-1243c1729527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:18.381876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1120/2216] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:18.393043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:18.394839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:18.406816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:18.470511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a9fb1-bd35-4c0b-b012-db22f96be0e7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:18.476182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:18.476849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:18.477134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:18.477412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:18.487726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:18.489399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b6b6c-bef4-4d26-b763-6c746d793e7e service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:18.499474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1097/2217] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:19.850409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:19.850947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:19.850947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:19.882692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] MFA Rules not processed for user `86f1f2722c5c4da59fd83036ce1700e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:19.884007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:19.959140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:19.980280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a9ec-0cd6-43e0-9183-660b9b3288a0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:19.983335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a9ec-0cd6-43e0-9183-660b9b3288a0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:19.983431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1121/2218] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:19.983523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a9ec-0cd6-43e0-9183-660b9b3288a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:20.001263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ec45a9ec-0cd6-43e0-9183-660b9b3288a0 None None] MFA Rules not processed for user `86f1f2722c5c4da59fd83036ce1700e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:20.003136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ec45a9ec-0cd6-43e0-9183-660b9b3288a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:20.015021 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1098/2219] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:42:20.026852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:20.028813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:20.107442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f1255-6681-4a20-9b8e-7f9cdaf483c6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:20.112951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:20.113621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:20.113907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:20.114211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:20.122825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:20.124494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:20.134680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1122/2220] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:20.774261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703e9d-8b8f-487b-acc1-7b82862753ec None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:20.774664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703e9d-8b8f-487b-acc1-7b82862753ec None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:20.775032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703e9d-8b8f-487b-acc1-7b82862753ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:20.819492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-61703e9d-8b8f-487b-acc1-7b82862753ec None None] MFA Rules not processed for user `86f1f2722c5c4da59fd83036ce1700e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:20.822644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-61703e9d-8b8f-487b-acc1-7b82862753ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:20.836051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1099/2221] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:27.085034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:27.086580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:27.169725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cea553-cb4c-40ba-afe4-b1a0b6ee46a4 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:27.176109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:27.177011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:27.177298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:27.177573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] PATH_INFO: `/v3/users/98826119296446c08dce72aff0a03c5d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:27.178919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98826119296446c08dce72aff0a03c5d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:27.180916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:27.233029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98826119296446c08dce72aff0a03c5d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:27.233686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98826119296446c08dce72aff0a03c5d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:27.240499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98826119296446c08dce72aff0a03c5d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:27.253154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98826119296446c08dce72aff0a03c5d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:27.263431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98826119296446c08dce72aff0a03c5d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:27.265737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:27.270891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1123/2222] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:27 2022] DELETE /identity/v3/users/98826119296446c08dce72aff0a03c5d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:27.275791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:27.276285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:27.276402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:27.295288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:27.296518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:27.361914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1100/2223] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:27.371407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:27.373659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:27.446147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1136d8-a9b6-45fe-9ad1-f2a07b21e866 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:27.453804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:27.454492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:27.454775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:27.455071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:27.468181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:27.469837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a97a-3586-4dc8-9426-9234cb7b8464 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:27.480159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1124/2224] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:27.875014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:27.876664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:27.904803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2b80f-4bf1-443d-a4b1-7581dafc3142 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:27.910463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:27.911718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:27.912040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:27.912323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] PATH_INFO: `/v3/projects/0d546952fa934a43a3443b91dfee479c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:27.913879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d546952fa934a43a3443b91dfee479c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:27.917557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.017505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d546952fa934a43a3443b91dfee479c', 'request_id': 'req-5922b453-ef64-465f-b5a4-a04d092fbcce'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:28.031778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d546952fa934a43a3443b91dfee479c', 'request_id': 'req-5922b453-ef64-465f-b5a4-a04d092fbcce'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:28.039945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d546952fa934a43a3443b91dfee479c', 'request_id': 'req-5922b453-ef64-465f-b5a4-a04d092fbcce'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:28.055176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] The token cache is being invalidate because project 0d546952fa934a43a3443b91dfee479c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:28.055760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:28.057724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5922b453-ef64-465f-b5a4-a04d092fbcce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:28.060731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1101/2225] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:27 2022] DELETE /identity/v3/projects/0d546952fa934a43a3443b91dfee479c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:28.065649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.066042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.066218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.083995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:28.085477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:28.154143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1125/2226] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:28.162307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.162682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.163106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.181818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:28.183163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:28.195517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1102/2227] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:28.197176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.207481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d64b8-a324-4e44-99c5-66894e7f8b1e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.212719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.213468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.213814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.214104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.214841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:28.215180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.216869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.229883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1126/2228] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:28.234940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.244539 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc006889-2e30-41bf-adb6-42fde9a9135a None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.249832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.250480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.250764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.251037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.251750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.253688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.266342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:28.293368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1103/2229] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:28 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:28.297638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.308570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc2757-08bd-4f81-92ea-824488f3c234 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.371474 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.372338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.372623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.372897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.373605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.375591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.399589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1127/2230] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:28 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:28.402068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.416290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2375-1c9e-4299-a8a7-e49e9f6d7d1f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.423132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.423922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.424207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.424637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.425405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.427731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.442186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1104/2231] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:28.446400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.457564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32557-93e0-4c57-96d1-db73bee63369 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.462465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.463183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.463469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.463743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.464508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.466201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3dc28-c66e-4ba8-8487-b56ac447cf16 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.477698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1128/2232] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:28.482266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.492470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e78865-c030-490b-a448-2820bbdd1dba admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.497165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.498055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.498335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.498612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] PATH_INFO: `/v3/projects/0b6b97613f5c4096a92500f182a3580c/users/c1f27713d08b453a979c8487f8071834/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.516297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6b97613f5c4096a92500f182a3580c, user_id=c1f27713d08b453a979c8487f8071834, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.519314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.547267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1105/2233] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:28 2022] PUT /identity/v3/projects/0b6b97613f5c4096a92500f182a3580c/users/c1f27713d08b453a979c8487f8071834/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:28.554963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.554963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.554963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.581801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:28.593230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] MFA Rules not processed for user `c1f27713d08b453a979c8487f8071834`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:28.595384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:28.694691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1129/2234] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:28.707902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.710769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:28.805402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eabc0-71f1-453b-84af-f45f82e0873b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.817318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.819893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.820435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.821591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.837545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.839321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf849af-8c91-4538-b9c4-6b01c291b63c service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:28.851799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1106/2235] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:28.881001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:28.953626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1debff-1d8f-4e55-8bd0-57782311c207 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:28.961131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:28.962183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:28.962495 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:28.962743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] PATH_INFO: `/v3/users/c1f27713d08b453a979c8487f8071834` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:28.964237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f27713d08b453a979c8487f8071834)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:28.968168 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:29.028152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f27713d08b453a979c8487f8071834'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:29.029562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f27713d08b453a979c8487f8071834'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:29.037073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f27713d08b453a979c8487f8071834'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:29.077587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f27713d08b453a979c8487f8071834'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:29.103416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f27713d08b453a979c8487f8071834'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:29.123654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1130/2236] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:28 2022] DELETE /identity/v3/users/c1f27713d08b453a979c8487f8071834 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:29.125657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:29.126028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:29.126398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:29.154008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:29.165168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:29.314657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1107/2237] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:29.329265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:29.330867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:29.369570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:29.417072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d493212-6674-4e34-a23b-96a7c36a0be9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:29.422487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:29.423255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:29.423640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:29.423919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:29.436898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:29.438883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffa03a-c16b-4e29-b978-718194b39d80 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:29.477211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1131/2238] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:29.936865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:29.938477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:29.953600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa90e6-8073-4cb3-8812-0aaa6a8f0dd7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:29.959546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:29.960570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:29.960950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:29.961299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] PATH_INFO: `/v3/projects/0b6b97613f5c4096a92500f182a3580c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:29.962854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6b97613f5c4096a92500f182a3580c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:29.965017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.028013 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6b97613f5c4096a92500f182a3580c', 'request_id': 'req-66af047b-c1bf-4505-a0bd-260829857888'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:30.032778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:30.034947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6b97613f5c4096a92500f182a3580c', 'request_id': 'req-66af047b-c1bf-4505-a0bd-260829857888'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:30.044354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6b97613f5c4096a92500f182a3580c', 'request_id': 'req-66af047b-c1bf-4505-a0bd-260829857888'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:30.056808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] The token cache is being invalidate because project 0b6b97613f5c4096a92500f182a3580c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:30.057196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:30.057916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-66af047b-c1bf-4505-a0bd-260829857888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:30.059698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1108/2239] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:29 2022] DELETE /identity/v3/projects/0b6b97613f5c4096a92500f182a3580c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:30.064672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.064965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.065267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.082451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:30.083695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:30.148696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1132/2240] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:30.157407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.157700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.157984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.177807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:30.180151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:30.189040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1109/2241] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:30.193139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.203752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e2cd9-d223-4d6e-b044-f07777b0f4d0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.208597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.209258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.209527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.209811 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.210562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:30.210872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.212526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.224702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1133/2242] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:30.229110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.238731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a0f90-f15f-4135-92da-946122e318bc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.243525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.244201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.244486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.244752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.245456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.247471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.277461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1110/2243] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:30 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:30.281929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.292631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6ef84-1846-4651-9240-d180c8fb257b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.360061 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.360756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.361042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.361319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.362339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.364530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.390320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1134/2244] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:30 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:42:30.394592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.406289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480c1f-3bd9-495f-8341-4a37c8398e8d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.411224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.411918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.412393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.412711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.413628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.415303 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.427134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1111/2245] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:42:30.431671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.442074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64211882-6261-4867-b13b-1959b88dda6a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.447312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.447991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.448308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.448559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.449302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.451284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35950813-9754-49fe-a8c0-fda277c6ab2e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.463169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1135/2246] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:30.467649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.481444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79d75d-a698-428f-af87-32a2af5587bf admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.486547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.487493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.487899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.488188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] PATH_INFO: `/v3/projects/c099988d7bdc415ab2f206cbe567e77b/users/a47eeee1d19f4deb9402314872f7ba30/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.503962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c099988d7bdc415ab2f206cbe567e77b, user_id=a47eeee1d19f4deb9402314872f7ba30, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.506635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.526209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233f44-2013-4073-8153-20966aea22e2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.527009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233f44-2013-4073-8153-20966aea22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.527322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233f44-2013-4073-8153-20966aea22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.528308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1112/2247] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:30 2022] PUT /identity/v3/projects/c099988d7bdc415ab2f206cbe567e77b/users/a47eeee1d19f4deb9402314872f7ba30/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:30.549531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5b233f44-2013-4073-8153-20966aea22e2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:30.550521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5b233f44-2013-4073-8153-20966aea22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:30.573472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:30.612382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1136/2248] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:42:30.622275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:30.624263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:30.695733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8906c2-73f3-40b4-8663-5540b0e8510d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:30.700282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:30.700928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:30.701353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:30.702301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:30.713270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:30.714714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:30.724964 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1113/2249] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:35.040095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.040691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.040691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.096542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] MFA Rules not processed for user `a47eeee1d19f4deb9402314872f7ba30`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:35.097912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:35.194031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1137/2250] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:35.198344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.209128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3ed3e-bf0a-40f4-a6ee-7c2ac0fbaaad service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.214123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.214902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.215295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.215716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.216598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.218613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.254039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1114/2251] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:35 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:35.258822 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.271763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc00c7-83e6-4990-b8e1-aba0e628c68a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.335513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.336201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.336487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.336764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.337460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.339479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.361694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1138/2252] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:35 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:35.365826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.376941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5ecb-671f-42b3-9384-e7e8267173f5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.381726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.382732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.383024 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.383342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.384085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.385708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.397345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1115/2253] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:35.402340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.414000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b2b1-c735-4c3b-a927-0fb4b8dfe991 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.420148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.421063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.421347 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.421627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] PATH_INFO: `/v3/projects/99ab2633ccd1468a91f91a6dc9c77809/users/f9ee526789ef432e81b18fc209f22ee9/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.439345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ab2633ccd1468a91f91a6dc9c77809, user_id=f9ee526789ef432e81b18fc209f22ee9, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.442126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.458900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1139/2254] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:35 2022] PUT /identity/v3/projects/99ab2633ccd1468a91f91a6dc9c77809/users/f9ee526789ef432e81b18fc209f22ee9/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:42:35.462883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.473366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76df8f-5295-4d70-bfaa-aaaef8d36825 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.537268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:35.560755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.561543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.561962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.562305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.563236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.564953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e2f42-fd65-4180-8c94-baf6f694bea2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.576194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1116/2255] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:35.580330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:35.591167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdadad2-2aa2-441b-82dd-2d3b84f70c18 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:35.595612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:35.596507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:35.596785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:35.597080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] PATH_INFO: `/v3/projects/99ab2633ccd1468a91f91a6dc9c77809/users/f9ee526789ef432e81b18fc209f22ee9/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:35.599130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ab2633ccd1468a91f91a6dc9c77809, user_id=f9ee526789ef432e81b18fc209f22ee9, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:35.601786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:35.614659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1140/2256] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:35 2022] PUT /identity/v3/projects/99ab2633ccd1468a91f91a6dc9c77809/users/f9ee526789ef432e81b18fc209f22ee9/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:40.839677 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33c79-8d29-41be-a22d-6da2ba9e082e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:40.840175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33c79-8d29-41be-a22d-6da2ba9e082e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:40.840255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33c79-8d29-41be-a22d-6da2ba9e082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:40.868877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-dee33c79-8d29-41be-a22d-6da2ba9e082e None None] MFA Rules not processed for user `f9ee526789ef432e81b18fc209f22ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:40.870110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dee33c79-8d29-41be-a22d-6da2ba9e082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:40.942917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1117/2257] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:40.958289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:40.960339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:40.975009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:41.074415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b71f3-43f7-480b-a9eb-d57c77276557 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:41.080198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:41.080875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:41.081154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:41.081437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:41.143437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:41.145390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:41.156316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1141/2258] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:42.495490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f642-ff6a-4a6e-9c6a-07639c1ab870 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:42.496028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f642-ff6a-4a6e-9c6a-07639c1ab870 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:42.496115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8f642-ff6a-4a6e-9c6a-07639c1ab870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:42.517984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f0c8f642-ff6a-4a6e-9c6a-07639c1ab870 None None] MFA Rules not processed for user `a47eeee1d19f4deb9402314872f7ba30`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:42.519456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f0c8f642-ff6a-4a6e-9c6a-07639c1ab870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:42.527488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1118/2259] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:51.151139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=52 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:52.511604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=44 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:54.259542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:54.263320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:54.424849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7537d-4408-4d78-a9bc-d00ed1183b13 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:54.429486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:54.430372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:54.430692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:54.430934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] PATH_INFO: `/v3/users/25d491b43061435a8fb61fd450ff9e6b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:54.432179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d491b43061435a8fb61fd450ff9e6b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:54.434104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:54.486821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d491b43061435a8fb61fd450ff9e6b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:54.487438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d491b43061435a8fb61fd450ff9e6b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:54.494613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d491b43061435a8fb61fd450ff9e6b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:54.506412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d491b43061435a8fb61fd450ff9e6b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:54.515865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d491b43061435a8fb61fd450ff9e6b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:54.523318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1142/2260] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:42:54 2022] DELETE /identity/v3/users/25d491b43061435a8fb61fd450ff9e6b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:54.528269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:54.528744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:54.528863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:54.557595 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:54.558900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:54.624619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1119/2261] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:54.635580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:54.637129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:54.663179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:54.743619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361d01-f069-47f5-a634-519aa46f26fb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:54.748112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:54.749053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:54.749434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:54.749789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:54.765630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:54.767140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85a151-9c65-42b3-a0a0-53e25056484a service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:54.785218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1143/2262] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:55.101177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.102986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:55.116482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5d0f6-2a42-4859-b222-a9786d75f0c1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.121634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.122525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.122985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.123318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] PATH_INFO: `/v3/projects/b8f18a3b6e6746108a836d285a840bc7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.124973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f18a3b6e6746108a836d285a840bc7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.127388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.183305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f18a3b6e6746108a836d285a840bc7', 'request_id': 'req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:55.189819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f18a3b6e6746108a836d285a840bc7', 'request_id': 'req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:55.198230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f18a3b6e6746108a836d285a840bc7', 'request_id': 'req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:55.211248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] The token cache is being invalidate because project b8f18a3b6e6746108a836d285a840bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:42:55.211543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:55.212210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-ab8ec450-7be3-4f25-9c43-301efdf1ec1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:42:55.213878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1120/2263] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:42:55 2022] DELETE /identity/v3/projects/b8f18a3b6e6746108a836d285a840bc7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:55.219069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.219370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.219651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.260925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:55.262370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:55.330859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1144/2264] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:42:55.338439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.338738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.339027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.358062 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:55.359430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:55.368875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1121/2265] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:55.376313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.388944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b4b6-5cde-4161-ad3f-128830f2e06d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.394151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.394828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.395142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.395420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.396158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:42:55.396463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.398086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.410838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1145/2266] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:42:55.416695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.426963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669fc6-0878-4e86-8073-2df759f7529f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.432949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.433717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.434036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.434414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.435312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.437634 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.469402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1122/2267] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:42:55 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:55.476584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.492482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769603d5-8eb5-4ae6-ad61-f327971adedc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.575631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:55.582237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.582913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.583223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.583501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.584334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.586326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.609402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1146/2268] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:42:55 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:42:55.611275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.621805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3114b2-8378-41c5-b611-c5129e0ddd50 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.626680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.627368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.627650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.627925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.628649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.630251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.644606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1123/2269] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:55.646063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.658655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f39c7-6cb6-44af-9eab-3b92ce538da5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.664785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.665601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.665894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.666316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.667107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.668781 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9159c6-a254-4573-9362-bf51510670bb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.681742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1147/2270] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:42:55.685809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.697328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf839ff0-e93d-49f4-8a44-a9115db9f2f0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.702240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.703174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.703456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.703731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] PATH_INFO: `/v3/projects/bd3ceb629d8d4c17a2578c614989e0cc/users/cc41ff7d5ee44eba98cc98101346ab29/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.718987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3ceb629d8d4c17a2578c614989e0cc, user_id=cc41ff7d5ee44eba98cc98101346ab29, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.721702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.736104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1124/2271] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:42:55 2022] PUT /identity/v3/projects/bd3ceb629d8d4c17a2578c614989e0cc/users/cc41ff7d5ee44eba98cc98101346ab29/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:42:55.746305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be33cd-2e88-4ca1-933e-63e435642223 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.746569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be33cd-2e88-4ca1-933e-63e435642223 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.746848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be33cd-2e88-4ca1-933e-63e435642223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.776739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d3be33cd-2e88-4ca1-933e-63e435642223 None None] MFA Rules not processed for user `cc41ff7d5ee44eba98cc98101346ab29`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:42:55.778431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d3be33cd-2e88-4ca1-933e-63e435642223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:55.848694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1148/2272] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:42:55.862287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:42:55.864517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:42:55.882039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:42:55.932402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46dc39-4424-447a-95be-9790ae10f0de admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:42:55.936572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:42:55.937236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:42:55.937523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:42:55.937795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:42:55.946650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:42:55.948122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:42:55.957289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1125/2273] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:42:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:43:05.845007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:05.953045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=20 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:26.467473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:26.468004 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:26.468093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:26.496877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:26.498087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:26.564170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1149/2274] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:26.605912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:26.607506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:26.681613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fa67-98c2-4bb3-9771-6ba45c7f93ae service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:26.687731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:26.688390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:26.688672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:26.688946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:26.697716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:26.699350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858044af-0afc-49c5-bd31-222883b2655d service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:26.707976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1126/2275] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:43:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:43:36.558030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:36.704383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=32 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:45.596247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:45.598017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:45.701705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fafbe-0a4d-4e31-ad1e-a5ae8160dea8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:45.709461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:45.710180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:45.710487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:45.710771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:45.712883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:45.726836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:45.728609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:45.740704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1150/2276] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:43:47.022974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.023490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.023603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.091890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:47.093193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:47.112526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1127/2277] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:47.125978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:47.147459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:47.150883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e9a68-adf1-4a0d-93f4-d66432067602 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:47.156374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:47.157253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.157373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.157574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.167738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:47.169116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec7fb-0f36-4ddf-87ff-e80a283ebf6b service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:47.178950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1151/2278] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:43:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:43:47.234330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee954f-7f2f-4b00-b395-8d3c1a92a6a1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:47.238883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:47.239796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.240085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.240364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] PATH_INFO: `/v3/users/cc41ff7d5ee44eba98cc98101346ab29` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.241577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc41ff7d5ee44eba98cc98101346ab29)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:47.243641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:47.300520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc41ff7d5ee44eba98cc98101346ab29'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.301177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc41ff7d5ee44eba98cc98101346ab29'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.309362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc41ff7d5ee44eba98cc98101346ab29'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.317739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc41ff7d5ee44eba98cc98101346ab29'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.330730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc41ff7d5ee44eba98cc98101346ab29'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.337777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1128/2279] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:43:47 2022] DELETE /identity/v3/users/cc41ff7d5ee44eba98cc98101346ab29 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:43:47.342559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.342964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.343150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.361958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:47.363294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:47.430771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1152/2280] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:43:47.440739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:47.442479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:47.490343 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:47.517590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a746ce-84c8-41a0-9713-e13bc70de0ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:47.521311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:47.522072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.522433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.522784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.540246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:47.541722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e30dee-6ead-4970-a80d-150f904f32e8 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:47.553493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1129/2281] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:43:47.850441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:47.852184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:47.866478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ba60-5c1f-4058-a888-1c12ebd846e7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:47.871758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:47.877487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.877789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.878075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] PATH_INFO: `/v3/projects/bd3ceb629d8d4c17a2578c614989e0cc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:47.879516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3ceb629d8d4c17a2578c614989e0cc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:47.881544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:47.933623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:47.937648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3ceb629d8d4c17a2578c614989e0cc', 'request_id': 'req-b4399add-39bf-4aa3-919a-13ab38d61b27'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.947272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3ceb629d8d4c17a2578c614989e0cc', 'request_id': 'req-b4399add-39bf-4aa3-919a-13ab38d61b27'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.954349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3ceb629d8d4c17a2578c614989e0cc', 'request_id': 'req-b4399add-39bf-4aa3-919a-13ab38d61b27'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.966132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] The token cache is being invalidate because project bd3ceb629d8d4c17a2578c614989e0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:43:47.966451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.967090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b4399add-39bf-4aa3-919a-13ab38d61b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:47.968817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1153/2282] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:43:47 2022] DELETE /identity/v3/projects/bd3ceb629d8d4c17a2578c614989e0cc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:43:47.974367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:47.974655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:47.974934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.005495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:48.006759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:48.071153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1130/2283] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:48.079415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.079698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.079997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.099250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:48.100497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:48.109231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1154/2284] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:48.113763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.124313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95410ab-d8d3-4aeb-84f4-27146f087837 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.129650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.130315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.130604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.130881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.131704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:43:48.132015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.133638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.147157 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1131/2285] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:43:48.152172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.165590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6041ed-4a04-423f-a21b-c26aeb8daea6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.173240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.174144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.174465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.174731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.175486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.177433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.212041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1155/2286] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:43:48 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:43:48.220261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.232499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a84622-9a54-4748-9fc5-d202a1f9d891 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.317660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.318355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.318628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.318906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.319647 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.322263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.343248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1132/2287] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:43:48 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:43:48.347542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.358694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24243df2-39d9-401f-b927-22b8e7825355 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.364130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.364943 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.365232 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.365531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.366272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.367959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.380772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1156/2288] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:43:48.385208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.396179 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70697-4084-437e-afbd-d5993ac4fee3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.401485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.402922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.403279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.403693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.404448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.406070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b6f64-b2cf-401c-86c5-89209259b658 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.422565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1133/2289] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:43:48.424163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.438835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7663-d436-4adc-8ccd-ce9c3de4d829 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.443854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.444807 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.445109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.445393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] PATH_INFO: `/v3/projects/46a9b3d4586841e0afa4c715b37aa3fb/users/8088aa46163c47d8b5636789c156b26f/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.462703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a9b3d4586841e0afa4c715b37aa3fb, user_id=8088aa46163c47d8b5636789c156b26f, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.466187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.481739 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1157/2290] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:43:48 2022] PUT /identity/v3/projects/46a9b3d4586841e0afa4c715b37aa3fb/users/8088aa46163c47d8b5636789c156b26f/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:43:48.492200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.492494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.492778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.528751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] MFA Rules not processed for user `8088aa46163c47d8b5636789c156b26f`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:48.529990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:48.575772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:48.619307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1134/2291] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:48.627888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:48.629814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:48.724116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1be8-45cc-451c-b09d-dba9438385ba admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:48.728727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:48.729592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:48.730442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:48.730785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:48.740584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:48.742254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:48.752740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1158/2292] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:43:49.185247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:49.188751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:49.313194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59578c-2180-407a-932a-bfc9a9a61efe None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:49.317873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:49.318791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:49.319304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:49.319416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] PATH_INFO: `/v3/users/f9aced17fa4848ecb0d5be31b87e1e5a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:49.320590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9aced17fa4848ecb0d5be31b87e1e5a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:49.322623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:49.372251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9aced17fa4848ecb0d5be31b87e1e5a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.372971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9aced17fa4848ecb0d5be31b87e1e5a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.380833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9aced17fa4848ecb0d5be31b87e1e5a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.389112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9aced17fa4848ecb0d5be31b87e1e5a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.401069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5a5eb8bc-fb4f-4ec9-a12d-e84ed7527cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9aced17fa4848ecb0d5be31b87e1e5a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.407478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1135/2293] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:43:49 2022] DELETE /identity/v3/users/f9aced17fa4848ecb0d5be31b87e1e5a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:43:49.616696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:49.648629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c428c4-68e7-4246-9d5b-8966bd406ac2 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:49.730740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:49.756220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:49.758611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:49.758611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:49.758917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] PATH_INFO: `/v3/projects/7df664c6311a4aefb792aa792ebc17fa` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:49.762852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df664c6311a4aefb792aa792ebc17fa)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:49.766246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:49.842926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df664c6311a4aefb792aa792ebc17fa', 'request_id': 'req-a266589a-fcca-4c82-b900-aaf1eae56aba'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.853171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df664c6311a4aefb792aa792ebc17fa', 'request_id': 'req-a266589a-fcca-4c82-b900-aaf1eae56aba'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.861624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df664c6311a4aefb792aa792ebc17fa', 'request_id': 'req-a266589a-fcca-4c82-b900-aaf1eae56aba'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.876101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] The token cache is being invalidate because project 7df664c6311a4aefb792aa792ebc17fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:43:49.876396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.877106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a266589a-fcca-4c82-b900-aaf1eae56aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:43:49.884974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1159/2294] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:43:49 2022] DELETE /identity/v3/projects/7df664c6311a4aefb792aa792ebc17fa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:43:49.887000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:49.887306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:49.887592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:49.912082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:49.913556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:49.974962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:49.994344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1136/2295] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:50.002249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.002873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.003290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.038441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:50.039720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:50.048001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1160/2296] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:50.052402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.063199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd9caa-3c25-43d7-8288-c6ffe0af4be0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.068032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.068731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.069015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.069288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.070017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:43:50.070336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.072000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.088151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1137/2297] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:43:50.089506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.099037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a05fbf-f225-4665-b9be-f4ec4bd021a2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.103790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.104632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.105450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.105450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.106107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.108157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.134955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1161/2298] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:43:50 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:43:50.139312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.149487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1df1-be95-40cc-80ec-1d04e991b430 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.208776 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.209512 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.209735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.210012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.210729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.212691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.235476 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1138/2299] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:43:50 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:43:50.239437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.251516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1774c-0246-4d52-bb50-561f3ca3cc57 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.256882 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.257563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.257800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.258100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.258830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.260818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.275548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1162/2300] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:43:50.276837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.291561 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ead02-a044-4db0-b504-238666aeac6f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.297122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.297803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.298097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.298362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.299136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.300736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a4907-0b85-47fb-bd68-32c4b981fa4f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.315123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1139/2301] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:43:50.317231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.334749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c323d-2eaf-462f-b1bd-bf03004404e9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.343085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.343085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.343085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.343085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] PATH_INFO: `/v3/projects/13dcb813f88146838e26dfb09c37d1a2/users/8a20fd58dc5c47269cf9afd93dfbcfe4/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.361036 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13dcb813f88146838e26dfb09c37d1a2, user_id=8a20fd58dc5c47269cf9afd93dfbcfe4, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.363946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.377685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1163/2302] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:43:50 2022] PUT /identity/v3/projects/13dcb813f88146838e26dfb09c37d1a2/users/8a20fd58dc5c47269cf9afd93dfbcfe4/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:43:50.389346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.389637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.389916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.418931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] MFA Rules not processed for user `8a20fd58dc5c47269cf9afd93dfbcfe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:43:50.420223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:50.492190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1140/2303] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:43:50.504192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:43:50.506037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:43:50.627318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a372-0e2a-47e4-be31-b5f9679d2796 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:43:50.631416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:43:50.632071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:43:50.632344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:43:50.632632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:43:50.643670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:43:50.645144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fac88-34f3-4d9a-bb38-aeefc09cb195 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:43:50.650115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:43:50.655584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1164/2304] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:00.488924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:03.070873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:03.072380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:03.151186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656c341-ffc3-4503-82cf-96ddef54d4b2 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:03.156734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:03.157966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:03.158255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:03.158538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] PATH_INFO: `/v3/users/b39c11b84e144804afc26d0ef9703022` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:03.160205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39c11b84e144804afc26d0ef9703022)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:03.165342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:03.218372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39c11b84e144804afc26d0ef9703022'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.219433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39c11b84e144804afc26d0ef9703022'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.229629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39c11b84e144804afc26d0ef9703022'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.237223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39c11b84e144804afc26d0ef9703022'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.249524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39c11b84e144804afc26d0ef9703022'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.255791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1141/2305] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:44:03 2022] DELETE /identity/v3/users/b39c11b84e144804afc26d0ef9703022 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:44:03.260800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:03.261299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:03.261493 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:03.280005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:03.281233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:03.347575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1165/2306] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:03.357819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:03.359253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:03.419085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829621c0-3fb5-46a7-b660-7ad595f386da admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:03.423042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:03.423778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:03.424056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:03.424347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:03.435318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:03.436785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef398-e33c-413b-8848-9a769f60e726 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:03.447975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1142/2307] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:03.831179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:03.832736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:03.852276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a039-3cac-4eec-baee-9fc4f8aa388b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:03.858152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:03.859175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:03.859344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:03.859702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] PATH_INFO: `/v3/projects/df15611c4bb346108d70f6a0c582ab84` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:03.863929 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df15611c4bb346108d70f6a0c582ab84)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:03.864285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:03.918377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df15611c4bb346108d70f6a0c582ab84', 'request_id': 'req-4ba1005a-14c2-4f03-b374-0731788381a0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.932312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df15611c4bb346108d70f6a0c582ab84', 'request_id': 'req-4ba1005a-14c2-4f03-b374-0731788381a0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.939973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df15611c4bb346108d70f6a0c582ab84', 'request_id': 'req-4ba1005a-14c2-4f03-b374-0731788381a0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.951139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] The token cache is being invalidate because project df15611c4bb346108d70f6a0c582ab84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:44:03.951446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.952132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4ba1005a-14c2-4f03-b374-0731788381a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:03.954103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1166/2308] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:44:03 2022] DELETE /identity/v3/projects/df15611c4bb346108d70f6a0c582ab84 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:44:03.958980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:03.960293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:03.960767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:03.980879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:03.982086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:03.999867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:04.060987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1143/2309] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:04.068927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.069331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.069729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.092533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:04.093877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:04.103157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1167/2310] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:04.108079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.119189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4927d6-6d6c-4268-9b7a-7bf5145c5a78 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.124102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.124773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.125170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.125473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.126940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:44:04.127304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.128991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.143617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1144/2311] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:44:04.151661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.161281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37f192-0bde-40cc-bca6-c6e0f11e2516 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.166438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.167119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.167403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.167685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.168385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.170311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.198093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1168/2312] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:44:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:44:04.203890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.215158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d560a-3c60-4de8-8200-3c6902080491 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.277885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.278651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.278934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.279239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.279955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.281963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.305204 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1145/2313] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:44:04 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:44:04.309981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.321303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edd5e-b5d1-4d17-80de-364e5bed7709 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.327142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.327796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.328077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.328406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.329420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.331038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.342486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1169/2314] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:44:04.346372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.357149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac9c8-8331-419f-84c1-7107a21c67cb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.363093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.363753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.364074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.364350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.365084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.366688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39fa26-a25e-46c1-b11a-523904f55575 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.377623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1146/2315] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:44:04.382429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.395527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d33b-91ac-47fa-9f9b-d8c0a8c38278 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.401457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.402354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.402637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.402901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] PATH_INFO: `/v3/projects/eb9beb4baa2b41808ba9ffa5f233ae48/users/797350f034d24e7580beeb67e6693a53/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.419587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9beb4baa2b41808ba9ffa5f233ae48, user_id=797350f034d24e7580beeb67e6693a53, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.422265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.438065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1170/2316] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:44:04 2022] PUT /identity/v3/projects/eb9beb4baa2b41808ba9ffa5f233ae48/users/797350f034d24e7580beeb67e6693a53/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:44:04.443518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.443854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.444131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.463374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:04.464604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:04.530787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1147/2317] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:04.540804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:04.542628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:04.569838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:04.611106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ed03-d881-45ab-b440-feb4e93d9ba6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:04.615850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:04.616536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:04.616816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:04.617089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:04.627610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:04.629132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dfbb-0df0-46d5-8816-e4b9e11d50a1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:04.640016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1171/2318] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:05.178895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:05.180611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.193583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0364e-2dcf-4010-be41-22175ae0ea21 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:05.199427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:05.200391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.200748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.201094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] PATH_INFO: `/v3/users/8a20fd58dc5c47269cf9afd93dfbcfe4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.202461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a20fd58dc5c47269cf9afd93dfbcfe4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:05.204828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:05.234542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:05.266639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a20fd58dc5c47269cf9afd93dfbcfe4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.267589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a20fd58dc5c47269cf9afd93dfbcfe4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.275345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a20fd58dc5c47269cf9afd93dfbcfe4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.283098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a20fd58dc5c47269cf9afd93dfbcfe4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.295961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a20fd58dc5c47269cf9afd93dfbcfe4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.304401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1148/2319] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:44:05 2022] DELETE /identity/v3/users/8a20fd58dc5c47269cf9afd93dfbcfe4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:44:05.309020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b44ec-ac20-4140-b50d-1187b144a762 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.309412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b44ec-ac20-4140-b50d-1187b144a762 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.309693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b44ec-ac20-4140-b50d-1187b144a762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.327205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-153b44ec-ac20-4140-b50d-1187b144a762 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:05.328442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-153b44ec-ac20-4140-b50d-1187b144a762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.402265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1172/2320] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:05.413130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:05.414924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.483014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de29c2-2acf-42f5-8dd1-c48f8881a60f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:05.487371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:05.488071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.488353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.488626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.499324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:05.500926 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:05.511734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1149/2321] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:05.690102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.690406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.690769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.725077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] MFA Rules not processed for user `797350f034d24e7580beeb67e6693a53`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:05.726561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.817693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1173/2322] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:05.829839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:05.831609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.856307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:05.858146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:05.877560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0474ed7-960d-4370-9c3a-bbb109e0e0bb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:05.883027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:05.884417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.884417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.885843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] PATH_INFO: `/v3/projects/13dcb813f88146838e26dfb09c37d1a2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.887113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13dcb813f88146838e26dfb09c37d1a2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:05.889073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:05.914064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352fb6b-95cf-42e2-a4a5-14832cf59034 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:05.918308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:05.918974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:05.919297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:05.919575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:05.932336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:05.933814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549beb2-8b0f-482e-a5e3-655916cefcd7 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:05.944016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1150/2323] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:05.949213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:05.950733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13dcb813f88146838e26dfb09c37d1a2', 'request_id': 'req-3d5e3014-bee6-48fa-ba95-67da1be14b81'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.960328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13dcb813f88146838e26dfb09c37d1a2', 'request_id': 'req-3d5e3014-bee6-48fa-ba95-67da1be14b81'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.969609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13dcb813f88146838e26dfb09c37d1a2', 'request_id': 'req-3d5e3014-bee6-48fa-ba95-67da1be14b81'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.981120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] The token cache is being invalidate because project 13dcb813f88146838e26dfb09c37d1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:44:05.981483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.982235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:05.984196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1174/2324] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:44:05 2022] DELETE /identity/v3/projects/13dcb813f88146838e26dfb09c37d1a2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:44:08.399806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb48cd-cf35-4fa9-a811-c31722c0c503 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:08.400376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb48cd-cf35-4fa9-a811-c31722c0c503 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:08.400376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb48cd-cf35-4fa9-a811-c31722c0c503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:08.420762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8fbb48cd-cf35-4fa9-a811-c31722c0c503 None None] MFA Rules not processed for user `797350f034d24e7580beeb67e6693a53`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:08.422159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8fbb48cd-cf35-4fa9-a811-c31722c0c503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:08.475308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:08.493279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1151/2325] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:14.712045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:14.713525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:14.786549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e3014-bee6-48fa-ba95-67da1be14b81 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:14.790297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:14.791042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:14.791402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:14.791868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:14.793920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:14.859522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:14.861220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:14.871329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1175/2326] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:18.487632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:23.644910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86061e9e-ae62-48e6-8b9a-a4a7aeb91da8 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:23.645530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86061e9e-ae62-48e6-8b9a-a4a7aeb91da8 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:23.645530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86061e9e-ae62-48e6-8b9a-a4a7aeb91da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:23.664905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-86061e9e-ae62-48e6-8b9a-a4a7aeb91da8 None None] MFA Rules not processed for user `86f1f2722c5c4da59fd83036ce1700e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:23.666030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-86061e9e-ae62-48e6-8b9a-a4a7aeb91da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:23.723338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1152/2327] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:24.867148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=57,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:33.716344 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=25 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:51.929760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:51.929760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:51.999293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faefb7d-e6a9-4068-9e4b-2b3b4cc582d3 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:52.004346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:52.005238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.005522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.005798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] PATH_INFO: `/v3/users/8088aa46163c47d8b5636789c156b26f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.007022 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8088aa46163c47d8b5636789c156b26f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:52.009055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:52.074996 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8088aa46163c47d8b5636789c156b26f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.075790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8088aa46163c47d8b5636789c156b26f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.083976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8088aa46163c47d8b5636789c156b26f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.097856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8088aa46163c47d8b5636789c156b26f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.107815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8088aa46163c47d8b5636789c156b26f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.120452 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1176/2328] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:44:51 2022] DELETE /identity/v3/users/8088aa46163c47d8b5636789c156b26f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:44:52.124976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.125511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.125619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.144761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:52.146123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:52.212106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1153/2329] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:52.222183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:52.223804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:52.287440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781de6-f0a9-46f9-a5d2-5093841fc0b2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:52.291313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:52.291974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.292377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.292675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.303395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:52.305044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869007f0-ca5e-40d0-b320-34950df53e52 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:52.316238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1177/2330] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:44:52.609487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:52.611029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:52.633314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66464d6-225d-4052-82a3-f71c53c8cab1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:52.639490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:52.640384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.640656 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.641080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] PATH_INFO: `/v3/projects/46a9b3d4586841e0afa4c715b37aa3fb` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.642579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a9b3d4586841e0afa4c715b37aa3fb)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:52.644753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:52.700382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a9b3d4586841e0afa4c715b37aa3fb', 'request_id': 'req-6728a9ba-d2cf-4e15-abe4-17d23630ca79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.707898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a9b3d4586841e0afa4c715b37aa3fb', 'request_id': 'req-6728a9ba-d2cf-4e15-abe4-17d23630ca79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.716665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a9b3d4586841e0afa4c715b37aa3fb', 'request_id': 'req-6728a9ba-d2cf-4e15-abe4-17d23630ca79'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.729854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] The token cache is being invalidate because project 46a9b3d4586841e0afa4c715b37aa3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:44:52.730200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.731019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6728a9ba-d2cf-4e15-abe4-17d23630ca79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:44:52.732864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1154/2331] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:44:52 2022] DELETE /identity/v3/projects/46a9b3d4586841e0afa4c715b37aa3fb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:44:52.738855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.739187 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.739446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.758815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:52.760203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:52.785074 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:52.843698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1178/2332] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:44:52.853499 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.853801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.854605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.890266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:52.891638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:52.904193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1155/2333] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:52.908932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:52.922483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0d197-d54b-498c-b4dd-ac12de768786 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:52.931273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:52.932159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.932351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.932592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.933379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:44:52.933687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:52.935434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:52.950309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1179/2334] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:44:52.955328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:52.967332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09636d-a6c3-4960-8aca-56bc145cb417 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:52.972153 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:52.972842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:52.973141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:52.973419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:52.974151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:52.976177 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.008959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1156/2335] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:44:52 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:44:53.013689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:53.024186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321f1fc-d580-4546-b65c-575c27227696 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:53.086827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:53.087646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.088038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.088411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.089184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:53.091896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.118475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1180/2336] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:44:53 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:44:53.122875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:53.134283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6ab45-18d0-40f0-8e0d-dd2f3d9e09ed admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:53.141400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:53.142748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.143247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.143748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.144603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:53.146506 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.158428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1157/2337] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:44:53.162846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:53.175300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda7399-4409-47a9-9f35-ee4a24c58264 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:53.180318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:53.181103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.181481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.181976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.182593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:53.184308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5920412-aeac-4adc-b53f-7a53b6fe1412 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.200354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1181/2338] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:44:53.201629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:53.212397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e57a8d-92b9-4d38-8adc-39c1c63282cc admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:53.216819 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:53.217840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.218124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.218469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] PATH_INFO: `/v3/projects/cee83a10226143fb857ff3a24c7bd618/users/34bd80b8b96244abb71e5de4319fc1d8/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.234557 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee83a10226143fb857ff3a24c7bd618, user_id=34bd80b8b96244abb71e5de4319fc1d8, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:53.237601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.256143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1158/2339] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:44:53 2022] PUT /identity/v3/projects/cee83a10226143fb857ff3a24c7bd618/users/34bd80b8b96244abb71e5de4319fc1d8/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:44:53.266842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc742dc5-cad1-419d-ac60-9dab5489d3de None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.267365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc742dc5-cad1-419d-ac60-9dab5489d3de None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.267920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc742dc5-cad1-419d-ac60-9dab5489d3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.297263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fc742dc5-cad1-419d-ac60-9dab5489d3de None None] MFA Rules not processed for user `34bd80b8b96244abb71e5de4319fc1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:53.298517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc742dc5-cad1-419d-ac60-9dab5489d3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:53.367922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1182/2340] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:44:53.379530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:44:53.381161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:53.404384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:44:53.445936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d7b6e-2b83-4e8b-959e-bdf6a1567c40 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:44:53.449949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:44:53.450696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:53.450985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:53.451299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:53.460933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:44:53.462387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12688b9e-59d4-4641-bd18-afbf45b8cf56 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:44:53.472085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1159/2341] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:44:55.912976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:44:55.913569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:44:55.913569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:44:55.933231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] MFA Rules not processed for user `34bd80b8b96244abb71e5de4319fc1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:44:55.934456 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:44:55.942314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1183/2342] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:03.467831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=17 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:05.927130 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:18.320648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eae68-0775-4428-ae25-ae6f1740738c None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:18.321152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eae68-0775-4428-ae25-ae6f1740738c None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:18.321260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eae68-0775-4428-ae25-ae6f1740738c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:18.338653 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-656eae68-0775-4428-ae25-ae6f1740738c None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:18.339891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-656eae68-0775-4428-ae25-ae6f1740738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:18.398598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1160/2343] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:45:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:18.459119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:18.461081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:18.545251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b79a9b-b5d2-44ab-ab8e-f6d8c7ae097e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:18.551355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:18.552054 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:18.552343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:18.552621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:18.561620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:18.563253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:18.572520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1184/2344] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:45:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:45:19.102790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e606-66e8-4f6d-b4f7-e8ab0cf7e9f9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.103110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e606-66e8-4f6d-b4f7-e8ab0cf7e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.103388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e606-66e8-4f6d-b4f7-e8ab0cf7e9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.121971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-0828e606-66e8-4f6d-b4f7-e8ab0cf7e9f9 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:19.124923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0828e606-66e8-4f6d-b4f7-e8ab0cf7e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:19.135165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1161/2345] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:19.148840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:19.160160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d94e8-8fe5-4d96-a56d-ad17aeb53f94 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:19.167002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:19.167715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.167997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.168271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.177894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:19.179600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:19.187865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1185/2346] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:45:19.398474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81458c-54b2-45a8-8b7a-22566a0890d9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.398969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81458c-54b2-45a8-8b7a-22566a0890d9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.399092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81458c-54b2-45a8-8b7a-22566a0890d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.417878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ce81458c-54b2-45a8-8b7a-22566a0890d9 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:19.419085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ce81458c-54b2-45a8-8b7a-22566a0890d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:19.427885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1162/2347] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:19.440969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:19.453377 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ca45f-1e96-447a-af75-bc0b791d77cd service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:19.459647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:19.460298 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.460585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.460864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.470221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:19.471870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:19.480213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1186/2348] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:45:19.833965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.834256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.834536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.852856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:19.854054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:19.862425 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1163/2349] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:19.875310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:19.886136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6c1f-23f5-473c-acf7-c087e9415512 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:19.892667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:19.893312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:19.893618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:19.893894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:19.902475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:19.904127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6410d09-8045-473f-8071-3c7a6a33b38f service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:19.912398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1187/2350] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:45:24.171342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:24.239065 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939169-29d7-4b16-9fca-e7a1864b28c1 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:24.245243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:24.246132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.246415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.246691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] PATH_INFO: `/v3/users/34bd80b8b96244abb71e5de4319fc1d8` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.248371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bd80b8b96244abb71e5de4319fc1d8)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:24.250365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:24.305635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bd80b8b96244abb71e5de4319fc1d8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.306495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bd80b8b96244abb71e5de4319fc1d8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.316596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bd80b8b96244abb71e5de4319fc1d8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.327636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bd80b8b96244abb71e5de4319fc1d8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.342732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bd80b8b96244abb71e5de4319fc1d8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.350954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1164/2351] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:45:24 2022] DELETE /identity/v3/users/34bd80b8b96244abb71e5de4319fc1d8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:45:24.355977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.357017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.357299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.377153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:24.378614 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:24.451925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1188/2352] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:24.461846 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:24.463398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:24.471143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:24.529391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892b9c1-6f94-472f-919a-3b3017d511ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:24.533185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:24.533907 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.534189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.534461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.544400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:24.545973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415ab1-bd6b-4b26-a026-8e1dcfcb2ead service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:24.556386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1165/2353] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:24.742608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:24.744484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:24.758279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a442bb-0b1c-4ed3-8d43-d9f097c1ac62 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:24.764934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:24.765846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.766136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.766415 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] PATH_INFO: `/v3/projects/cee83a10226143fb857ff3a24c7bd618` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.767703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee83a10226143fb857ff3a24c7bd618)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:24.769639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:24.819215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:24.830157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee83a10226143fb857ff3a24c7bd618', 'request_id': 'req-8db50a9f-d048-4153-92cf-500034cf39a3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.839140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee83a10226143fb857ff3a24c7bd618', 'request_id': 'req-8db50a9f-d048-4153-92cf-500034cf39a3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.850079 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee83a10226143fb857ff3a24c7bd618', 'request_id': 'req-8db50a9f-d048-4153-92cf-500034cf39a3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.865911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] The token cache is being invalidate because project cee83a10226143fb857ff3a24c7bd618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:45:24.866244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.867006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8db50a9f-d048-4153-92cf-500034cf39a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:45:24.868738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1189/2354] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:45:24 2022] DELETE /identity/v3/projects/cee83a10226143fb857ff3a24c7bd618 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:45:24.874110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.874484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.874760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.897532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:24.899266 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:24.964494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1166/2355] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:24.973848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:24.974207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:24.974541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:24.994084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:24.995333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:25.003559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1190/2356] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:25.007866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.018465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3b056-8065-4a9d-a240-879cf984cf75 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.023619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.024092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.024429 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.024661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.026070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:45:25.026393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.028066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.046162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1167/2357] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:45:25.047767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.057654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120a4a0-e2c5-4f24-ba5f-58775bc83e9c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.063567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.064233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.064510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.064778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.065476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.067801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.113733 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1191/2358] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:45:25 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:45:25.117912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.129322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc4f92-672e-40ef-a6fd-9497f89c1139 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.189937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.190870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.191545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.192056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.192922 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.194933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.217247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1168/2359] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:45:25 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:45:25.221566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.236941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acba115-0179-46fd-b70c-b2d7ce2db8d0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.237691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.238428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.238763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.239041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.239858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.241556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.256936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1192/2360] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:45:25.258277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.271619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ca044-c7b8-4f48-a0d3-1e9cd25f326f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.278714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.279572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.279957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.280617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.281589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.283914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696464fb-cd45-40a1-8dab-b68ab287cabd admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.297224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1169/2361] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:45:25.301903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.312554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6bac3-52b3-48e5-ac9a-a1aaba3424f9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.317133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.318022 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.318300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.318571 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] PATH_INFO: `/v3/projects/256a9b5f5ad1405bb9fe21d591ef677d/users/d162fcafed2c4ae099a5786cceb84891/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.333897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256a9b5f5ad1405bb9fe21d591ef677d, user_id=d162fcafed2c4ae099a5786cceb84891, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.336621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.353199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1193/2362] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:45:25 2022] PUT /identity/v3/projects/256a9b5f5ad1405bb9fe21d591ef677d/users/d162fcafed2c4ae099a5786cceb84891/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:45:25.358684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812501-33c3-44be-98e2-bddae9595084 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.358970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812501-33c3-44be-98e2-bddae9595084 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.359282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812501-33c3-44be-98e2-bddae9595084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.380702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bb812501-33c3-44be-98e2-bddae9595084 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:25.382198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bb812501-33c3-44be-98e2-bddae9595084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:25.400666 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:25.459938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1170/2363] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:25.471753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:25.473786 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:25.546399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494cb70-9412-43c8-85bc-7f1acf8cc809 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:25.551218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:25.551954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:25.552306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:25.552654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:25.562934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:25.564524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd51db-6cd1-46dc-9960-6eab44271c99 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:25.574331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1194/2364] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:26.691401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:26.692008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:26.692008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:26.710929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] MFA Rules not processed for user `b49313559188427bb9f42068583a8292`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:26.712281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:26.776369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1171/2365] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:31.359549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:31.360422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:31.360842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:31.393732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] MFA Rules not processed for user `d162fcafed2c4ae099a5786cceb84891`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:31.395564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:31.477769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:31.506891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1195/2366] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:31.510975 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:31.534554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84eff-0f5e-471e-a97d-c4ccfcec38da None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:31.544292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:31.545283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:31.545748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:31.546180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:31.547101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:31.549225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:31.617837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1172/2367] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:45:31 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:45:31.621565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:31.636628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dcbc8d-328b-48df-9d20-33da1edb61b5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:31.778468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:31.779711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:31.780426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:31.781345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:31.782427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:31.785462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:31.815378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1196/2368] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:45:31 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:45:31.818848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:31.833591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a2e9d-29ce-494c-9383-105debe5b7b0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:31.840875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:31.842086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:31.842589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:31.843246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:31.844384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:31.846984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:31.864720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1173/2369] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:45:31.868792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:31.879769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3e9fa-6ae1-446b-a7b8-48c0849defff admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:31.885873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:31.887039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:31.887514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:31.888336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] PATH_INFO: `/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c/users/be1f4af420984092a27be3409b162ab4/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:31.910577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dc3dea81894b6ab41ae93f1c48693c, user_id=be1f4af420984092a27be3409b162ab4, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:31.913693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:31.939134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1197/2370] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:45:31 2022] PUT /identity/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c/users/be1f4af420984092a27be3409b162ab4/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:45:31.940345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:31.953988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0d4d6-ec49-4b95-9021-e29189dc780e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:32.031618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:32.032600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:32.032871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:32.033142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:32.033893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:32.035741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:32.055092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1174/2371] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:45:32.058490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:32.070552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b29aee-21bb-4522-87d9-5aa99d95e39e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:32.076029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:32.077150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:32.077437 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:32.077713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] PATH_INFO: `/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c/users/be1f4af420984092a27be3409b162ab4/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:32.080003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dc3dea81894b6ab41ae93f1c48693c, user_id=be1f4af420984092a27be3409b162ab4, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:32.082545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:32.100469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1198/2372] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:45:32 2022] PUT /identity/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c/users/be1f4af420984092a27be3409b162ab4/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:45:32.771988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:32.774085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:32.805406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:32.847964 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0411f5-e896-46ff-90dd-983691b0d472 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:32.919842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:32.922664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:32.922664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:32.922664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:32.924304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:33.031482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:33.033657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d24db-2600-4908-b8ce-3287a187dbd3 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:33.062896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1175/2373] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:33.112506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:33.125812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d775210-dd7e-45dd-b60e-ba6dd9d8a8d0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:33.130024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:33.131299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:33.131764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:33.132216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:33.134318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:33.153616 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:33.155576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:33.188012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1199/2374] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:37.799333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:37.799874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:37.799874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:37.834703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] MFA Rules not processed for user `be1f4af420984092a27be3409b162ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:45:37.835981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:37.903250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:37.927405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1176/2375] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:45:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:45:37.944045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:37.947593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:37.963381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13debf-a237-4e08-aa1f-1656c3566af7 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:37.969724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:37.970386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:37.970736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:37.971151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:38.033101 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:38.034736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:38.044109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1200/2376] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:40.832674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:40.834398 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:40.852742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b37cef-42cf-4643-b285-3ed6c8c97005 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:40.861370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:40.862046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:40.862336 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:40.862615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:40.864551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:40.934185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:40.936160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180aa98-ce35-4c7e-8415-44bcb3946512 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:40.946866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1177/2377] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:45:44.564623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:45:44.576808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b8e0e-ae14-4492-b45c-98aa9b13868f service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:45:44.584599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:45:44.584599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:45:44.584813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:45:44.585175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:45:44.587198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:45:44.618015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:44.658245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:45:44.660385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:45:44.673773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1201/2378] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:45:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:45:50.942043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=35 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:45:54.667125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=15 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:46:02.809084 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302fa28-29a6-43b6-8c6d-8307daf63009 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:02.809504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302fa28-29a6-43b6-8c6d-8307daf63009 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:02.809731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302fa28-29a6-43b6-8c6d-8307daf63009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:02.841125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7302fa28-29a6-43b6-8c6d-8307daf63009 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:02.842389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7302fa28-29a6-43b6-8c6d-8307daf63009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:02.914703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1178/2379] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:02.927208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:02.928777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:03.006715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b3935-9638-4143-9deb-7f462b51bbbd service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:03.012125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:03.012777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:03.013065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:03.015439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:03.024019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:03.025599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:03.034147 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1202/2380] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:46:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:46:12.173596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:12.174269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:12.174362 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:12.194862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] MFA Rules not processed for user `797350f034d24e7580beeb67e6693a53`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:12.196183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:12.277034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1179/2381] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:46:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:12.517053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:12.527735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adaaa-cd89-40c4-b276-a1691b2555f2 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:12.551126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:12.551126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:12.551126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:12.551126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:12.551802 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:12.560866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:12.562727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:12.572006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1203/2382] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:12.917555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:12.919801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:13.012218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176a2f8-9cf7-495f-b4ae-a99721c04b0b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:13.017380 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:13.018064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:13.018345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:13.018950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:13.020810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:13.034102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:13.035893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:13.046186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1180/2383] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:15.221159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:15.231937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca16c8-803e-4a0a-a977-353408786f52 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:15.236477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:15.237121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:15.237590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:15.238207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:15.240502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:15.251424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:15.253239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c8d1e-807b-454d-bd18-e52f7919bba3 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:15.263278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1204/2384] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:16.032568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:16.050989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae685-bdbd-43f5-8628-c3418a747303 service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:16.070213 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:16.070655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:16.071032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:16.071342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:16.073183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:16.091367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:16.093466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:16.108091 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1181/2385] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:18.807574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae379f-b82f-4fca-be59-1905b810ffac None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:18.808104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae379f-b82f-4fca-be59-1905b810ffac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:18.808104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae379f-b82f-4fca-be59-1905b810ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:18.831429 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-56ae379f-b82f-4fca-be59-1905b810ffac None None] MFA Rules not processed for user `d162fcafed2c4ae099a5786cceb84891`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:18.832846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-56ae379f-b82f-4fca-be59-1905b810ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:18.840855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1205/2386] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:26.102830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:46:27.038260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:27.055582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5207-9e7e-4f43-8416-59fd4252f6f7 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:27.069730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:27.070402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.070688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.070963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.072969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:27.086214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:27.087943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f6c3c-73a6-4361-aa49-50ad45531fae service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:27.099206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1182/2387] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:27.425677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.426279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.426279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.427850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1206/2388] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:46:27.434072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8d94-7aba-4be1-b5c9-4fda42e98ee6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.434573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8d94-7aba-4be1-b5c9-4fda42e98ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.434800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8d94-7aba-4be1-b5c9-4fda42e98ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.474979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-912f8d94-7aba-4be1-b5c9-4fda42e98ee6 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:27.476286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-912f8d94-7aba-4be1-b5c9-4fda42e98ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:27.487146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1183/2389] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:27.499273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:27.501308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:27.508823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=48 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:46:27.583499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6368912-3c5d-4824-8532-f70810a32fd5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:27.587825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:27.588592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.589005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.589384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.601986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:27.603620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a589793-df9b-4d15-bbaa-340a152d5fa2 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:27.618619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1207/2390] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:27.691987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec78f7-2db1-4e9a-a3bc-d7bfe619a6bc None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.692298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec78f7-2db1-4e9a-a3bc-d7bfe619a6bc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.692574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec78f7-2db1-4e9a-a3bc-d7bfe619a6bc None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.693977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1184/2391] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:46:27.699107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276237cb-10d4-4888-b868-6b07c66677a4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.699401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276237cb-10d4-4888-b868-6b07c66677a4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.699678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276237cb-10d4-4888-b868-6b07c66677a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.718602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-276237cb-10d4-4888-b868-6b07c66677a4 None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:27.719706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-276237cb-10d4-4888-b868-6b07c66677a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:27.726897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1208/2392] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:27.738909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec78f7-2db1-4e9a-a3bc-d7bfe619a6bc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:27.750736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec78f7-2db1-4e9a-a3bc-d7bfe619a6bc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:27.754583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:27.755243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.755529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.755809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.763359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:27.765112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28b3c-7c75-45e2-b865-5d1b0ba864bc service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:27.773005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1185/2393] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:46:27.855569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46adca-fb7d-4b9a-b56b-29db40066173 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.855855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46adca-fb7d-4b9a-b56b-29db40066173 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.856129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46adca-fb7d-4b9a-b56b-29db40066173 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.857550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1209/2394] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:46:27.862747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe0bf3-e4d5-461d-8f7a-a6cfd88389b7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.863079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe0bf3-e4d5-461d-8f7a-a6cfd88389b7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.863455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe0bf3-e4d5-461d-8f7a-a6cfd88389b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.881839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-cabe0bf3-e4d5-461d-8f7a-a6cfd88389b7 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:27.883233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-cabe0bf3-e4d5-461d-8f7a-a6cfd88389b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:27.892785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1186/2395] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:27.903218 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46adca-fb7d-4b9a-b56b-29db40066173 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:27.916092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46adca-fb7d-4b9a-b56b-29db40066173 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:27.919965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:27.920620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:27.920907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:27.921188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:27.932724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:27.934163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:27.945515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1210/2396] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:46:33.122997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:33.123531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:33.123531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:33.142210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:46:33.143431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:46:33.151265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1187/2397] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:46:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:46:33.173889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:46:33.190746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2228011-8a18-453d-89cb-aa246463936c service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:46:33.197946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:46:33.198607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:46:33.198890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:46:33.199194 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:46:33.208926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:46:33.210525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:46:33.218921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1211/2398] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:46:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:46:43.136685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=27 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:46:43.215480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=40 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:47:18.730126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:18.739509 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82015d7a-d3df-4e6e-a5ef-12920cdc72bc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:18.743290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:18.744012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:18.744364 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:18.744751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:18.754580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:18.756151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aabd6-e576-454a-adab-d6012f23d1f5 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:18.767123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1188/2399] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:47:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:47:20.681525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:20.700658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f86901-85e2-425d-a30b-e6ef823d9155 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:20.706432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:20.707207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:20.707663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:20.708017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:20.709862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:20.723102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:20.724711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:20.736146 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1212/2400] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:47:23.674454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:23.674978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:23.675228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:23.695302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:47:23.696530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:23.704427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1189/2401] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:47:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:47:23.717721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:23.730369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5124-31eb-4220-8684-47dacd5cb35e service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:23.735986 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:23.736630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:23.736911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:23.737188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:23.746029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:23.747647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:23.756137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1213/2402] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:47:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:47:33.689779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=9 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:47:33.752715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=12 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:47:54.629437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:54.643262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd83f8-acd6-47e0-b098-d12cebe973eb None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:54.649436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:54.650200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:54.650482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:54.650757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:54.652911 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:54.665029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:54.666687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:54.685450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:54.686921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1190/2403] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:47:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:47:54.687229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:54.757057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa4b1-988a-49c4-beeb-ec1def81ccee service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:54.761978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:54.762852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:54.763178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:54.763471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] PATH_INFO: `/v3/users/797350f034d24e7580beeb67e6693a53` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:54.764687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] RBAC: Authorizing `identity:delete_user(user_id=797350f034d24e7580beeb67e6693a53)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:54.766791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:54.826840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797350f034d24e7580beeb67e6693a53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:54.827808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797350f034d24e7580beeb67e6693a53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:54.838375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797350f034d24e7580beeb67e6693a53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:54.850680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797350f034d24e7580beeb67e6693a53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:54.859176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f3554592-2a42-4f72-9c7c-23cad237855e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797350f034d24e7580beeb67e6693a53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:54.865383 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1214/2404] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:47:54 2022] DELETE /identity/v3/users/797350f034d24e7580beeb67e6693a53 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:47:55.054404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.067289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc48bc-1d0a-4a72-8936-1303f4da6a2d service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.133515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.135068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.135388 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.135672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] PATH_INFO: `/v3/projects/eb9beb4baa2b41808ba9ffa5f233ae48` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.136900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9beb4baa2b41808ba9ffa5f233ae48)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.138831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.200758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9beb4baa2b41808ba9ffa5f233ae48', 'request_id': 'req-4e362ff1-05bb-47c7-84c6-f1f315c46c58'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:55.207778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9beb4baa2b41808ba9ffa5f233ae48', 'request_id': 'req-4e362ff1-05bb-47c7-84c6-f1f315c46c58'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:55.216564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9beb4baa2b41808ba9ffa5f233ae48', 'request_id': 'req-4e362ff1-05bb-47c7-84c6-f1f315c46c58'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:55.228959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] The token cache is being invalidate because project eb9beb4baa2b41808ba9ffa5f233ae48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:47:55.229368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:55.230052 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:47:55.231685 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1191/2405] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:47:55 2022] DELETE /identity/v3/projects/eb9beb4baa2b41808ba9ffa5f233ae48 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:47:55.236905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493b91-4b6d-4b51-8dde-0c66e0278957 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.237215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493b91-4b6d-4b51-8dde-0c66e0278957 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.237497 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493b91-4b6d-4b51-8dde-0c66e0278957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.262280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-82493b91-4b6d-4b51-8dde-0c66e0278957 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:47:55.263552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-82493b91-4b6d-4b51-8dde-0c66e0278957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.324777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.326513 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.332432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1215/2406] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:47:55.341137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.341454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.341728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.361329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:47:55.362593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.377336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1216/2407] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:47:55.381687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.392240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de871fd7-3948-421c-99f9-89037ea12bd8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.397451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.398106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.398387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.398661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.399572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:47:55.399890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.401605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.407198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ff1-05bb-47c7-84c6-f1f315c46c58 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.410922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:47:55.413615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.414307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.414591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.414917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.417235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.418916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1217/2408] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:47:55.419794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.430458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748c27-c7fe-40a1-8739-985862bf6ff8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.432318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:47:55.436279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.436966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.437237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.437533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.438242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.440301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.479540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1218/2409] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:47:55 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:47:55.482034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.483605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.483786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.494003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1192/2410] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:47:55.550964 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad564ff-1372-466f-bace-38f5b20aac0f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.556438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.557245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.557578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.557995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.558764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.561226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.583485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1219/2411] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:47:55 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:47:55.589955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.601814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca282c-0cd2-4e66-a8d2-3f38252632f9 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.615245 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.616051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.616414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.616762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.617532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.619884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.631835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1193/2412] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:47:55.638694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.650725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42c21-a207-4703-b575-85c79d7e54c4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.656447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.657125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.657409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.657686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.658423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.660072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb11d7-6bce-4ba5-b0ab-cd90bf51c9b2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.671644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1220/2413] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:47:55.675947 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.688487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53cb5d-0fbb-4193-9939-b29e21f7091b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.693788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.694875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.695196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.695518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] PATH_INFO: `/v3/projects/8bb810c5032e4bc8a92597ab38dc8f97/users/49ed7cea2ae245a3984e58c075ff5333/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.713452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb810c5032e4bc8a92597ab38dc8f97, user_id=49ed7cea2ae245a3984e58c075ff5333, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.716369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.734831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1194/2414] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:47:55 2022] PUT /identity/v3/projects/8bb810c5032e4bc8a92597ab38dc8f97/users/49ed7cea2ae245a3984e58c075ff5333/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:47:55.746305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.746853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.746853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.776112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] MFA Rules not processed for user `49ed7cea2ae245a3984e58c075ff5333`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:47:55.777338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.845980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1221/2415] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:47:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:47:55.857702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:47:55.859630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:47:55.918004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01461107-c6ca-4a8e-97fb-5745872e8483 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:47:55.921808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:47:55.922463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:47:55.922742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:47:55.923020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:47:55.932849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:47:55.934413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c7c8d-3be9-4a84-8d50-8d45af640744 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:47:55.944086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1195/2416] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:47:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:05.843287 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=79,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:05.939702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:21.607428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:21.677153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a661c-bb68-4159-a033-c8cb18cba120 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:21.682199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:21.683161 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:21.683446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:21.683747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] PATH_INFO: `/v3/users/49ed7cea2ae245a3984e58c075ff5333` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:21.685039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ed7cea2ae245a3984e58c075ff5333)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:21.687467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:21.742159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ed7cea2ae245a3984e58c075ff5333'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:21.742848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ed7cea2ae245a3984e58c075ff5333'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:21.749942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ed7cea2ae245a3984e58c075ff5333'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:21.761853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ed7cea2ae245a3984e58c075ff5333'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:21.771203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ed7cea2ae245a3984e58c075ff5333'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:21.777325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1222/2417] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:48:21 2022] DELETE /identity/v3/users/49ed7cea2ae245a3984e58c075ff5333 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:48:21.782101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:21.782636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:21.782740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:21.800481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:48:21.801693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:21.868262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1196/2418] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:48:21.878226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:21.879820 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:21.948386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9aff9-5eff-4ec8-a6e4-7f839a075480 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:21.958574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:21.959299 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:21.959569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:21.959839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:21.970734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:21.972395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb60420-6bb7-4427-a11d-de79346ab6a4 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:21.995343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1223/2419] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:22.042168 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:22.044143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.056920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3069d7-7e93-4f60-8144-abaf4f4cdb96 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:22.061158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:22.062016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.062318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.062597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.064757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.078307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:22.079834 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c05978-de73-47bc-b15b-f75efbf0063c service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:22.088400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1197/2420] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:48:22.196527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade9043-c16c-428f-bacf-9eb460c1ebf4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.197027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade9043-c16c-428f-bacf-9eb460c1ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.197395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade9043-c16c-428f-bacf-9eb460c1ebf4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.199285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1224/2421] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:48:22.205728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.206018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.206299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.225340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] MFA Rules not processed for user `7db42ab9d8ff4cd2931f40085f983216`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:48:22.226563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.233106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1198/2422] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:48:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:48:22.245635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade9043-c16c-428f-bacf-9eb460c1ebf4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:22.256787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade9043-c16c-428f-bacf-9eb460c1ebf4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:22.260637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:22.261359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.261521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.261793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.269773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:22.271234 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89611a-262a-419e-a1f1-421979cbbe01 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:22.281314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1225/2423] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:22.363603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:22.365508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.377288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764373-5780-4968-9b2d-465096a2137c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.377595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764373-5780-4968-9b2d-465096a2137c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.377865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38764373-5780-4968-9b2d-465096a2137c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.379475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1226/2424] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:48:22.388607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.388894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.389189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.389321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa15c7-6b33-4c64-aab3-ed63e966a8bc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:22.401323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:22.402221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.402529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.403410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] PATH_INFO: `/v3/projects/8bb810c5032e4bc8a92597ab38dc8f97` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.411180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb810c5032e4bc8a92597ab38dc8f97)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:22.412617 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:48:22.413164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:22.413888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.418460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:22.467421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb810c5032e4bc8a92597ab38dc8f97', 'request_id': 'req-10fec791-e44f-4f8c-a527-5ab41e0286c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:22.474126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb810c5032e4bc8a92597ab38dc8f97', 'request_id': 'req-10fec791-e44f-4f8c-a527-5ab41e0286c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:22.482085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb810c5032e4bc8a92597ab38dc8f97', 'request_id': 'req-10fec791-e44f-4f8c-a527-5ab41e0286c3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:22.491361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1227/2425] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:48:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:48:22.494212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] The token cache is being invalidate because project 8bb810c5032e4bc8a92597ab38dc8f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:48:22.494511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:22.495220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:48:22.497034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1199/2426] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:48:22 2022] DELETE /identity/v3/projects/8bb810c5032e4bc8a92597ab38dc8f97 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:48:22.502642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:22.513673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9d2ca-5383-45b9-bd4b-9671f5b935c3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:22.577314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:22.578090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:22.578376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:22.578656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:22.580852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:22.665087 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:22.666603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:22.679419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1228/2427] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:23.339478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:23.341271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:23.360139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fec791-e44f-4f8c-a527-5ab41e0286c3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:23.367085 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:23.367792 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:23.368059 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:23.368348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:23.370373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:23.444003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:23.445613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885830a5-1792-4663-b479-9cbe5b2f861f service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:23.455565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1200/2428] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:32.673982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:33.450830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:46.076580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:48:46.078850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:46.204649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee2229-b937-49f5-a488-38db927cdeff service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:48:46.214844 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:48:46.215570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:46.216027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:46.216478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:46.219418 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:46.239590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:48:46.242492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd7329-0aa3-4b1f-b28b-61943e74cb77 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:48:46.267893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1229/2429] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:48:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:48:55.438753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2b9ba-426a-4600-aa95-8b11862ca696 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:55.439654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2b9ba-426a-4600-aa95-8b11862ca696 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:55.439885 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2b9ba-426a-4600-aa95-8b11862ca696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:55.459040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-52c2b9ba-426a-4600-aa95-8b11862ca696 None None] MFA Rules not processed for user `86f1f2722c5c4da59fd83036ce1700e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:48:55.460338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-52c2b9ba-426a-4600-aa95-8b11862ca696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:55.467620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1201/2430] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:48:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:48:56.252883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:48:58.205838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:48:58.206704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:48:58.206704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:48:58.224434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] MFA Rules not processed for user `b49313559188427bb9f42068583a8292`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:48:58.225690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:48:58.284606 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1230/2431] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:48:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:49:05.453225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:49:08.276649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=25 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:49:20.595431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:49:20.595977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:49:20.595977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:49:20.614734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] MFA Rules not processed for user `80113e82c4a74f4e8fd42b88aca3920d`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:49:20.615982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:49:20.677261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1202/2432] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:49:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:49:30.669729 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=25 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:32.484741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:32.486540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:32.500584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6636eb1-711c-4932-a1b5-b0ab0a7a0a08 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:32.505144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:32.505793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:32.506080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:32.506359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:32.508231 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:32.521518 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:32.523392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771837f-8899-47e7-b035-0ef5b1f7c20d service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:32.534886 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1231/2433] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:35.608006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:35.609458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:35.622760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642a784-78a1-41de-8e1a-2a4e6c1ba91e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:35.627280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638766-1039-4752-839c-428b259bb703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:35.628033 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638766-1039-4752-839c-428b259bb703 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:35.628379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638766-1039-4752-839c-428b259bb703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:35.628879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638766-1039-4752-839c-428b259bb703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:35.630841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-84638766-1039-4752-839c-428b259bb703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:35.715149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84638766-1039-4752-839c-428b259bb703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:35.716742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84638766-1039-4752-839c-428b259bb703 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:35.727878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1203/2434] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:36.325340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:36.325866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:36.325866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:36.344238 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] MFA Rules not processed for user `d162fcafed2c4ae099a5786cceb84891`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:36.345529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:36.404428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1232/2435] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:38.274167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638766-1039-4752-839c-428b259bb703 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:38.288906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638766-1039-4752-839c-428b259bb703 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:38.295009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:38.299628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:38.300824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:38.301452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:38.301830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:38.306973 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:38.309841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4c03d-2f54-4be0-aa97-4e5229b79da3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:38.315300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:38.316278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:38.316783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:38.317272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:38.328922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:38.331177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:38.343985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:38.345942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1233/2436] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:50:38.348766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:38.365138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1204/2437] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:38.512595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:38.523107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd187b1-de99-4cde-a9ca-781a79054900 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:38.527015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:38.527691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:38.528176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:38.528609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:38.530893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:38.548262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:38.549937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:38.568205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1234/2438] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:43.214558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:43.216784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:43.234845 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c36b-0d54-4a88-a336-7127688435d0 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:43.241322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:43.242511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:43.242854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:43.243468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:43.245863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:43.260529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:43.262535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:43.286918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1205/2439] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:43.333523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:43.336170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:43.362348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6748497-ca2a-4f01-a674-ea828f411765 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:43.368686 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:43.369912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:43.370414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:43.370908 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] PATH_INFO: `/v3/users/86f1f2722c5c4da59fd83036ce1700e1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:43.372913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f1f2722c5c4da59fd83036ce1700e1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:43.375673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:43.445136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f1f2722c5c4da59fd83036ce1700e1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.446276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f1f2722c5c4da59fd83036ce1700e1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.455988 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f1f2722c5c4da59fd83036ce1700e1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.477469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f1f2722c5c4da59fd83036ce1700e1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.495035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c2caaaa6-19d5-4cb4-abe2-3bf2e27cf272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f1f2722c5c4da59fd83036ce1700e1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.505685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1235/2440] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:50:43 2022] DELETE /identity/v3/users/86f1f2722c5c4da59fd83036ce1700e1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:43.739172 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:43.750791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd38b4-3266-4b51-91ed-d781d31ec65f service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:43.836511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:43.837387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:43.837665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:43.837937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] PATH_INFO: `/v3/projects/0907423ad5f14bf9a0680c9cdb78c995` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:43.839152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0907423ad5f14bf9a0680c9cdb78c995)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:43.841063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:43.889014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0907423ad5f14bf9a0680c9cdb78c995', 'request_id': 'req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.894785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0907423ad5f14bf9a0680c9cdb78c995', 'request_id': 'req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.904441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0907423ad5f14bf9a0680c9cdb78c995', 'request_id': 'req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.912396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:43.919632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] The token cache is being invalidate because project 0907423ad5f14bf9a0680c9cdb78c995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:50:43.920391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.921560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-f2340426-b68e-4efe-9e4c-aa09aaf12cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:43.923575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1206/2441] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:50:43 2022] DELETE /identity/v3/projects/0907423ad5f14bf9a0680c9cdb78c995 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:43.928658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:43.928963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:43.929249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:43.947845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:43.949062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:43.987024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:44.017237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1236/2442] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:44.025507 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.025802 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.026080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.045808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:44.047237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:44.055921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1207/2443] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:44.060165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.069963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c1a6-f146-4c53-a0bc-3abaed040ee7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.074773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.075621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.075991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.076331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.077143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:50:44.077520 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.079263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.091567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1237/2444] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:50:44.095680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.105113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce604d-e49c-4d54-b4cc-b042b36ff625 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.113053 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.113815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.114176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.114523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.115366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.117409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.151730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1208/2445] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:50:44 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:50:44.155897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.166972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110764-1e6e-4b00-9153-6e6265b4c88e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.237274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.238915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.239317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.239595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.240280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.242217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.264292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1238/2446] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:50:44 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:50:44.268320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.279966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e33bfd-b0af-48d0-aa7a-0efc13af9a56 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.284939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.285641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.285936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.286211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.287097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.288741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.300412 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1209/2447] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:50:44.304136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.315269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e35b-49de-499a-bbd7-07861aa6247f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.320009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.320692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.320956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.321227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.321952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.323585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517f25a-b16c-42f3-8e64-efcda32f2f75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.335283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1239/2448] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:50:44.339416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.351496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaac01-c0e8-4048-8662-ddb92d181051 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.356601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.357518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.357809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.358122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] PATH_INFO: `/v3/projects/992e429cb62643a1885c29538604ae5d/users/d5d5414645ec47e1bcaa56860ba6daf6/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.374263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=992e429cb62643a1885c29538604ae5d, user_id=d5d5414645ec47e1bcaa56860ba6daf6, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.376893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.392445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1210/2449] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:50:44 2022] PUT /identity/v3/projects/992e429cb62643a1885c29538604ae5d/users/d5d5414645ec47e1bcaa56860ba6daf6/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:44.398931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.399286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.399563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.419276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:44.420525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:44.513030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1240/2450] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:44.536571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:44.538751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:44.609001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d242-82c9-4537-8a89-dec0f816dd97 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:44.612865 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:44.613664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:44.614010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:44.614357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:44.625034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:44.626874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:44.638617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1211/2451] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:50:45.447564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:45.449648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:45.463592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5fb07-8f0d-42bc-b3c3-c44ed8b0204a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:45.468638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:45.469656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:45.470007 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:45.470362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] PATH_INFO: `/v3/users/a47eeee1d19f4deb9402314872f7ba30` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:45.471741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47eeee1d19f4deb9402314872f7ba30)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:45.473696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:45.548762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:45.557878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47eeee1d19f4deb9402314872f7ba30'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:45.558559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47eeee1d19f4deb9402314872f7ba30'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:45.566864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47eeee1d19f4deb9402314872f7ba30'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:45.583311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47eeee1d19f4deb9402314872f7ba30'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:45.593440 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47eeee1d19f4deb9402314872f7ba30'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:45.600731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1241/2452] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:50:45 2022] DELETE /identity/v3/users/a47eeee1d19f4deb9402314872f7ba30 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:45.926529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:45.942693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e1baa-e3c4-44a1-bd31-6ef092677be1 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:46.015697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:46.017218 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:46.017825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:46.018356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] PATH_INFO: `/v3/projects/c099988d7bdc415ab2f206cbe567e77b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:46.020415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c099988d7bdc415ab2f206cbe567e77b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:46.022678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:46.071067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:46.080852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c099988d7bdc415ab2f206cbe567e77b', 'request_id': 'req-75acdb16-020c-4cbf-89e7-54dd6ccabe90'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.088984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c099988d7bdc415ab2f206cbe567e77b', 'request_id': 'req-75acdb16-020c-4cbf-89e7-54dd6ccabe90'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.099603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c099988d7bdc415ab2f206cbe567e77b', 'request_id': 'req-75acdb16-020c-4cbf-89e7-54dd6ccabe90'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.112176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] The token cache is being invalidate because project c099988d7bdc415ab2f206cbe567e77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:50:46.112466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.113117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.114931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1212/2453] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:50:45 2022] DELETE /identity/v3/projects/c099988d7bdc415ab2f206cbe567e77b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:46.118678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:46.120514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:46.135280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde73b9-0752-460f-a8c9-4b2631907245 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:46.200151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:46.201343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:46.201621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:46.201893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] PATH_INFO: `/v3/users/f9ee526789ef432e81b18fc209f22ee9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:46.203216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ee526789ef432e81b18fc209f22ee9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:46.205872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:46.260989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ee526789ef432e81b18fc209f22ee9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.262044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ee526789ef432e81b18fc209f22ee9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.270193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ee526789ef432e81b18fc209f22ee9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.291836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ee526789ef432e81b18fc209f22ee9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.301902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e3f5470a-54e3-43f6-a00b-14bd78386b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ee526789ef432e81b18fc209f22ee9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.317546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1242/2454] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:50:46 2022] DELETE /identity/v3/users/f9ee526789ef432e81b18fc209f22ee9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:46.510494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:46.511960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:46.528211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acdb16-020c-4cbf-89e7-54dd6ccabe90 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:46.622141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:46.623102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:46.623385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:46.623652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] PATH_INFO: `/v3/projects/99ab2633ccd1468a91f91a6dc9c77809` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:46.625765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ab2633ccd1468a91f91a6dc9c77809)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:46.628711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:46.702984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ab2633ccd1468a91f91a6dc9c77809', 'request_id': 'req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.711219 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ab2633ccd1468a91f91a6dc9c77809', 'request_id': 'req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.720874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ab2633ccd1468a91f91a6dc9c77809', 'request_id': 'req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.734514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] The token cache is being invalidate because project 99ab2633ccd1468a91f91a6dc9c77809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:50:46.734915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.735587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:50:46.737276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1213/2455] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:50:46 2022] DELETE /identity/v3/projects/99ab2633ccd1468a91f91a6dc9c77809 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:50:49.408326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e126c-b7ec-4d91-8d88-d61852086112 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:49.408977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e126c-b7ec-4d91-8d88-d61852086112 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:49.408977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e126c-b7ec-4d91-8d88-d61852086112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:49.438809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c3e126c-b7ec-4d91-8d88-d61852086112 None None] MFA Rules not processed for user `d5d5414645ec47e1bcaa56860ba6daf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:49.440125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c3e126c-b7ec-4d91-8d88-d61852086112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:49.467384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:49.528169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1243/2456] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:49.540493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:49.542338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:49.604721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:50:49.626913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe955d8-f6bc-4b4f-bb51-bd68924e798f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:49.632331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:49.632991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:49.633275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:49.633553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:49.641828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:49.643545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:49.653241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1214/2457] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:50:51.049040 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:51.050221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:51.050221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:51.070123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] MFA Rules not processed for user `d5d5414645ec47e1bcaa56860ba6daf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:50:51.071533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:51.078469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1244/2458] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:50:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:50:59.534707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:50:59.546768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fa785-34a4-4a67-b476-1ff7d3d27ec1 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:50:59.554108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:50:59.554774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:50:59.555088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:50:59.555379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:50:59.557342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:50:59.644054 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:50:59.645673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d116635-b6e2-4583-ac35-7f44d5130982 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:50:59.674999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1215/2459] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:50:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:01.063801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=19 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:05.496931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:05.511392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d25e3-e490-442b-b5cc-f34a3d0cfc94 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:05.517914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:05.518581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:05.518850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:05.519170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:05.521519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:05.621728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:05.623864 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:05.646386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1245/2460] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:51:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:09.671151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=49 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:15.642843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=41 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:16.434345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:16.434917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:16.434917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:16.464427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] MFA Rules not processed for user `b49313559188427bb9f42068583a8292`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:51:16.465767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:16.474103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1216/2461] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:51:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:51:17.206798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:17.208461 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:17.285663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a9c0e-8c9b-4b74-b4f8-2dadf4b300d2 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:17.291430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:17.292108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:17.292422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:17.292706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:17.294672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:17.306695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:17.308334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:17.319345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1246/2462] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:51:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:19.312436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:19.314426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:19.375889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258125f-97c8-41c7-9605-61efb6976d3f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:19.380058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:19.380978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:19.381379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:19.381769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:19.384049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:19.396116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:19.397578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea24180-e073-494a-8e1f-fffabd8d5ab0 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:19.406756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1217/2463] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:51:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:27.314794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:29.402919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=32 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:30.432271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:30.454993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03eabde-f866-40ea-ae21-7c3dbb676d46 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:30.460810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:30.461489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:30.461778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:30.462059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:30.465241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:30.477481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:30.479118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:30.491399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1247/2464] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:33.449972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93514f-81a8-44c5-a6d5-72c3a8b07412 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:33.450610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93514f-81a8-44c5-a6d5-72c3a8b07412 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:33.450704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93514f-81a8-44c5-a6d5-72c3a8b07412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:33.470035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5d93514f-81a8-44c5-a6d5-72c3a8b07412 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:51:33.471304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5d93514f-81a8-44c5-a6d5-72c3a8b07412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:33.532421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1218/2465] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:51:33.582685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:33.584325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:33.656612 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601185de-2117-44b7-ae45-ca733de20b16 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:33.663323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:33.664289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:33.664569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:33.664842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:33.673683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:33.675322 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf485da-12ce-4ca4-b7f3-d7ae9b6b46dd service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:33.683607 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1248/2466] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:51:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:51:43.525137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=25 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:43.680253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=38 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:51:50.796953 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:50.797570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:50.797570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] PATH_INFO: `/` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:50.799000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1219/2467] 10.222.0.74 () {54 vars in 921 bytes} [Thu Jun 16 16:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:51:50.805018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18a9-7863-433d-832d-436e79ba9cbf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:50.805573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18a9-7863-433d-832d-436e79ba9cbf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:50.805674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd18a9-7863-433d-832d-436e79ba9cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:50.827062 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f7cd18a9-7863-433d-832d-436e79ba9cbf None None] MFA Rules not processed for user `b6297dc028a44750a69132257e0aede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:51:50.828301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f7cd18a9-7863-433d-832d-436e79ba9cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:50.838678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1249/2468] 10.222.0.74 () {58 vars in 1018 bytes} [Thu Jun 16 16:51:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:51:50.851235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:50.853205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:50.915454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65ce6-f3f6-4fe9-9fb9-eb4291b3bc08 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:50.919365 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:50.920146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:50.920521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:50.921072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:50.931800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:50.935142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:50.944575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1220/2469] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:51:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:51:52.867262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c543-ca11-45bc-bfeb-b52de34f425d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:52.867793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c543-ca11-45bc-bfeb-b52de34f425d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:52.868066 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c543-ca11-45bc-bfeb-b52de34f425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:52.887663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4d62c543-ca11-45bc-bfeb-b52de34f425d None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:51:52.889175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4d62c543-ca11-45bc-bfeb-b52de34f425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:51:52.898084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1250/2470] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:51:52.910460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:51:52.922934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1ca39-742c-48ee-8d43-7cd607dc6105 service glance] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:51:52.928217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:51:52.928868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:51:52.929152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:51:52.929432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:51:52.937944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:51:52.939631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:51:52.947939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1221/2471] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:51:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:52:02.882205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:52:02.944552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=32 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:52:17.103913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:52:17.104531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:52:17.104531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:52:17.123724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:52:17.125005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:52:17.132645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1251/2472] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:52:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:52:17.146105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:52:17.158399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593df54d-5d53-499d-a49c-10bf1a28fb98 service swift] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:52:17.164668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:52:17.165332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:52:17.165618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:52:17.165895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:52:17.175838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:52:17.177457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e3fa2-b66f-4be2-ba8b-6e3f5e9b32ed service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:52:17.186170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1222/2473] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:52:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:52:27.118220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:52:27.182800 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=6 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:52:27.585281 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:52:27.600253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce43ecb-0095-411c-afe6-07b110ebb3a9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:52:27.607191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:52:27.607836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:52:27.608121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:52:27.608393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:52:27.610566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:52:27.623554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:52:27.625235 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:52:27.635918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1252/2474] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:52:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:52:37.631797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:06.718982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:06.719574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:06.719574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:06.749984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] MFA Rules not processed for user `d162fcafed2c4ae099a5786cceb84891`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:06.751259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:06.836567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1223/2475] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:16.834382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=37 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:27.200099 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:27.202841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:27.277024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d756894-ce61-4117-8640-4b9fd043e749 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:27.281608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:27.282683 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:27.283525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:27.283873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:27.285757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:27.365047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:27.366438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:27.376019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1253/2476] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:29.047852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:29.068284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27982d-3bd6-487e-858c-71a6db897659 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:29.073848 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:29.074516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:29.074788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:29.075120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:29.077118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:29.090081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:29.092138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:29.103654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1224/2477] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:37.371855 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=55 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:39.098832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=9 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:46.525893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:46.549376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ce4f8-d85a-4977-bde9-a94f4fba08ca service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:46.555140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:46.555847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:46.556133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:46.556411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:46.559290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:46.586593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:46.588289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:46.599005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1254/2478] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:50.472946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:50.483694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aa5c7-4eff-4a38-a452-fa18eeaa967a service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:50.487766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:50.488503 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:50.488818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:50.489101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:50.491293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:50.592607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:50.594161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:50.614078 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1225/2479] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:52.058500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:52.068876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebae3b-a4c1-4317-83a2-4950dff3f46e service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:52.072598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:52.073245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:52.073533 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:52.073809 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:52.076344 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:52.089252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:52.091077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:52.101425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1255/2480] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:55.040979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:55.045752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:55.063005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a01e4e-1e05-49ee-b0eb-61d54af82f0e service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:55.070009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:55.071252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.071659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.072056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] PATH_INFO: `/v3/users/b49313559188427bb9f42068583a8292` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.073449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49313559188427bb9f42068583a8292)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:55.077328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:55.137418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49313559188427bb9f42068583a8292'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.138193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49313559188427bb9f42068583a8292'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.147645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49313559188427bb9f42068583a8292'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.156076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49313559188427bb9f42068583a8292'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.170317 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bd3fa104-e29b-41d0-9718-fba823c78713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49313559188427bb9f42068583a8292'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.176750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1226/2481] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:53:55 2022] DELETE /identity/v3/users/b49313559188427bb9f42068583a8292 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:55.375693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:55.389871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d634a8e-99bc-4453-96cf-8d2a5b17efc7 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:55.448800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:55.449797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.450083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.450360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] PATH_INFO: `/v3/projects/8725edcf74cc463cafe3614a3bd880e6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.451597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8725edcf74cc463cafe3614a3bd880e6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:55.453516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:55.503966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8725edcf74cc463cafe3614a3bd880e6', 'request_id': 'req-0e2288c3-f174-4391-97f3-42daacbca9ff'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.512768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8725edcf74cc463cafe3614a3bd880e6', 'request_id': 'req-0e2288c3-f174-4391-97f3-42daacbca9ff'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.519500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8725edcf74cc463cafe3614a3bd880e6', 'request_id': 'req-0e2288c3-f174-4391-97f3-42daacbca9ff'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.530223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] The token cache is being invalidate because project 8725edcf74cc463cafe3614a3bd880e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:53:55.530656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.531392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-0e2288c3-f174-4391-97f3-42daacbca9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:55.533580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1256/2482] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:53:55 2022] DELETE /identity/v3/projects/8725edcf74cc463cafe3614a3bd880e6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:53:55.540444 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.542609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.543505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.606393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:55.609786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:55.611818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:55.702305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1227/2483] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:55.710159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.710428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.710902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.733499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:55.735487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:55.749923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1257/2484] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:55.754455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:55.766329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276ee57-2b13-45cd-81b6-4c5ca67450fd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:55.773239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:55.774035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.774415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.774783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.775882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:53:55.776439 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:55.778176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:55.800648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:55.801874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1228/2485] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:53:55.824773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fec94-e693-4493-8bf0-0aca26139554 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:55.829672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:55.830578 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.830759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.831190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.832023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:55.834047 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:55.863169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1258/2486] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:53:55 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:53:55.867405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:55.880232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc3c80-108b-445a-9bd3-087bc094e45d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:55.963699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:55.964072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:55.964447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:55.964856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:55.965928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:55.967982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:55.994624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1229/2487] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:53:55 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:55.999307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.009860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6638c8-8c13-4244-9bc8-b6839abbaeb2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.015127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.019829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.020032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.020283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.021122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.022946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.036950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1259/2488] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:53:56.040992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.051473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d850c-aaf0-46a5-814f-cf4a3f85c9c4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.056009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.056680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.056962 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.057238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.057960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.059734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed8705c-8a94-4d3c-bde7-eb3ca84df74a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.070342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1230/2489] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:56.075226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.082041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:56.086170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be839-54fa-4583-a25d-7268b96c896c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.092738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.093841 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.094191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.094480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] PATH_INFO: `/v3/projects/744ee1c94ffb4553a0bcf84cf433defa/users/a025691cdaaa4654b5e8a23532eec54a/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.111253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744ee1c94ffb4553a0bcf84cf433defa, user_id=a025691cdaaa4654b5e8a23532eec54a, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.114946 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.131292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1260/2490] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:56 2022] PUT /identity/v3/projects/744ee1c94ffb4553a0bcf84cf433defa/users/a025691cdaaa4654b5e8a23532eec54a/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:53:56.151205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.151625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.152016 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.204136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] MFA Rules not processed for user `a025691cdaaa4654b5e8a23532eec54a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:56.207150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.300296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1231/2491] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:56.304923 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.399853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4c722-5626-45ca-8a43-baa8529b42e8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.407510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.408230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.408513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.408837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.409579 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.412257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.420563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.424451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.435531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:56.450905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1261/2492] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:53:56 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:56.454741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.548502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f7eb-5a22-4234-a8df-d6538e2c7a2b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.551747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824fe5d-98b6-456f-9537-c58049a480fd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.554241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.555428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.555789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.556157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.557001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.559100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.582240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1262/2493] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:53:56 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:53:56.583462 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.587590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.600059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d1463-b0ca-4be3-91ff-aa0650693bca admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.604623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.605499 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.605848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.606125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] PATH_INFO: `/v3/users/80113e82c4a74f4e8fd42b88aca3920d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.607480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80113e82c4a74f4e8fd42b88aca3920d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.609390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.620902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.622112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.622401 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.622706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.624758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.637703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.639636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.650353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1232/2494] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:56.651447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.662443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80113e82c4a74f4e8fd42b88aca3920d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.663232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb54e4-b660-47df-938d-b0ac11668b46 service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.667445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80113e82c4a74f4e8fd42b88aca3920d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.668931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.669522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.669804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.670017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:56.670161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.670821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.672453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.677033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80113e82c4a74f4e8fd42b88aca3920d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.690106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1233/2495] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:56.692662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80113e82c4a74f4e8fd42b88aca3920d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.694221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.696590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.703551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80113e82c4a74f4e8fd42b88aca3920d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.710166 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24943f-c1e6-422f-8eb4-1cfcb165ed72 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.710310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1263/2496] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:53:56 2022] DELETE /identity/v3/users/80113e82c4a74f4e8fd42b88aca3920d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:56.771751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.772655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.772941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.773215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] PATH_INFO: `/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5/users/1280b485b0454379a22ebd0ff7c5cea0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.788031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d80873b0c5e45f487b5f93d3b9ab0a5, user_id=1280b485b0454379a22ebd0ff7c5cea0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.790715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.804714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1234/2497] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:56 2022] PUT /identity/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5/users/1280b485b0454379a22ebd0ff7c5cea0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:56.808894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.818713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88cb2f-6dee-4d6c-b1fb-ba905f7a107b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.880708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.881407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.881706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.881969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.886304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.887593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.887750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.889895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.904104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1264/2498] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:56.904293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03d3ca-6504-43eb-b8ba-c2f41a9396e4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.905588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:56.909233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.910113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.910406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.910691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] PATH_INFO: `/v3/projects/627133a3363742fda58273b067fab968` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.912260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=627133a3363742fda58273b067fab968)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.914223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.915998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:56.916366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5631df-b92e-4936-969b-24fa285ff7ba admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:56.922513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:56.923458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.923738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.924010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] PATH_INFO: `/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5/users/1280b485b0454379a22ebd0ff7c5cea0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.926124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d80873b0c5e45f487b5f93d3b9ab0a5, user_id=1280b485b0454379a22ebd0ff7c5cea0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:56.928925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834319f2-1be4-4a51-94d0-5c57f1e609ea admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:56.940428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1265/2499] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:56 2022] PUT /identity/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5/users/1280b485b0454379a22ebd0ff7c5cea0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:53:56.951805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:56.952103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:56.952378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:56.969475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627133a3363742fda58273b067fab968', 'request_id': 'req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.975075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627133a3363742fda58273b067fab968', 'request_id': 'req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.979093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] MFA Rules not processed for user `1280b485b0454379a22ebd0ff7c5cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:56.980595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:56.984190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627133a3363742fda58273b067fab968', 'request_id': 'req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.995693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] The token cache is being invalidate because project 627133a3363742fda58273b067fab968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:53:56.996092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.996904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b196410c-3ba2-473f-ad30-ab3b7ab05fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:56.998916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1235/2500] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:53:56 2022] DELETE /identity/v3/projects/627133a3363742fda58273b067fab968 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:53:57.003777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2291e-80b6-4674-a41b-098e2993a5fb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.004185 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2291e-80b6-4674-a41b-098e2993a5fb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.004467 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2291e-80b6-4674-a41b-098e2993a5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.023441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-73b2291e-80b6-4674-a41b-098e2993a5fb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:57.024651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-73b2291e-80b6-4674-a41b-098e2993a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.051305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1266/2501] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:53:57.063309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.064995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.088198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1236/2502] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:57.095862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.096154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.096432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.106968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:57.115628 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:57.116849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.125876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1237/2503] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:53:57.130290 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.139734 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5b509-74e2-45d9-b2ee-c29b4ccafcab None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.144296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.144932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.145217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.145494 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.146223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:53:57.146533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.148180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.151247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b6347-9a9a-44bc-aef6-84a120f4e3ee None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.155044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.155705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.155985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.156256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.160237 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1238/2504] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:53:57.164419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.169052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.170544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.174190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa7a89-dc94-4cdb-b785-a395579c28df admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.179023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.179697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.179979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.180175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1267/2505] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:57.180536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.181233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.183152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.212421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1239/2506] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:53:57 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:57.216037 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.226975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c52d5-8e40-481a-b8f1-0262f3780430 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.255180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.286346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.287026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.287335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.287613 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.288307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.290412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.309708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:57.311255 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1268/2507] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:53:57 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:57.315372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.315517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784d15c-3f68-4cf3-bd78-2022ff8177ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.321767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.322725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.323070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.323617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.326424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04279bce-2fe0-465d-8291-32f03ff89e89 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.326605 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.331900 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.332750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.333131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.333446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.334222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.335873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.346624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1269/2508] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:53:57.350691 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.360570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada780-3356-4c92-8244-9b98385f9a54 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.365310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.365962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.366245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.366523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.367286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.368878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.379688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1270/2509] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:53:57.383694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.390447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.391940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c6a1-4571-4c4b-97c7-75e80683e8cb service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.393577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be8ae0-2b17-441e-870c-f646d9ac895e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.398318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.399297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.399576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.399847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] PATH_INFO: `/v3/projects/865f6562c3704721aff6b41b28d075d4/users/5ab61dd4bdcd4fbca82887335b237707/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.401479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1240/2510] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:57.415942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865f6562c3704721aff6b41b28d075d4, user_id=5ab61dd4bdcd4fbca82887335b237707, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.418774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.432812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1271/2511] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:57 2022] PUT /identity/v3/projects/865f6562c3704721aff6b41b28d075d4/users/5ab61dd4bdcd4fbca82887335b237707/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:53:57.443635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.443929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.444212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.474955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] MFA Rules not processed for user `5ab61dd4bdcd4fbca82887335b237707`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:57.476301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.556110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1241/2512] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:53:57.567815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.569638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:57.637690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:57.654792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cda8f-fdf9-43da-8532-46c4e990a311 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.660554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.661288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.661646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.661993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.670420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.672057 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e791d52-d248-4a55-8a96-9ab976d86ea1 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:57.681081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1272/2513] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:57.893697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:57.972814 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9f97-d172-4280-98d4-6a158fa22962 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:57.981951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:57.983443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:57.983871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:57.984361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] PATH_INFO: `/v3/users/5ab61dd4bdcd4fbca82887335b237707` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:57.986410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab61dd4bdcd4fbca82887335b237707)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:57.989639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:58.039629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:58.050445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab61dd4bdcd4fbca82887335b237707'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.051122 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab61dd4bdcd4fbca82887335b237707'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.059723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab61dd4bdcd4fbca82887335b237707'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.066665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab61dd4bdcd4fbca82887335b237707'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.079813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab61dd4bdcd4fbca82887335b237707'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.086406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1242/2514] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:53:57 2022] DELETE /identity/v3/users/5ab61dd4bdcd4fbca82887335b237707 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:58.091827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.092120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.092399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.114504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:58.115759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:58.179572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1273/2515] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:53:58.192468 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:58.194637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:58.262590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c7315-1c51-4e0d-9c8c-23f61501ad73 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:58.268104 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:58.268757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.269038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.269312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.279335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:58.280779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41814e0-17cf-4c7c-9d00-033225d942c6 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:58.290456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1243/2516] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:53:58.581743 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:58.583947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:58.596494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a122-1a45-45bc-9286-ed51e89f6f6e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:58.601697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:58.602576 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.602897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.603157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] PATH_INFO: `/v3/projects/865f6562c3704721aff6b41b28d075d4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.604834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] RBAC: Authorizing `identity:delete_project(project_id=865f6562c3704721aff6b41b28d075d4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:58.606835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:58.663228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865f6562c3704721aff6b41b28d075d4', 'request_id': 'req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.671875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865f6562c3704721aff6b41b28d075d4', 'request_id': 'req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.680005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865f6562c3704721aff6b41b28d075d4', 'request_id': 'req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.691732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] The token cache is being invalidate because project 865f6562c3704721aff6b41b28d075d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:53:58.692106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.692902 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5bd93b5d-fb6d-4ab4-a386-5a0acfb5594c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:53:58.694665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1274/2517] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:53:58 2022] DELETE /identity/v3/projects/865f6562c3704721aff6b41b28d075d4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:58.699829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.700169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.700573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.719678 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:58.720981 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:58.784965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1244/2518] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:58.793954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.794247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.794526 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.813992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:58.815444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:58.825193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1275/2519] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:58.829740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:58.840275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e5cc6-8011-48fc-9bb5-f843821d9af0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:58.845486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:58.846416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.846811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.847460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.848377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:53:58.848795 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:58.850684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:58.866163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1245/2520] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:53:58.870603 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:58.880975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de1a31-21ed-4665-83bb-dcc84adeb40f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:58.887107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:58.888948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:58.889759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:58.890588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:58.891625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:58.893955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:58.932625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1276/2521] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:53:58 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:58.934180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:58.944987 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b9cd8-a922-4c11-819f-f1a587d065dc admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:58.989674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:59.007799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.008480 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.008768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.009048 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.009736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.012279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.034828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1246/2522] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:53:58 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:59.038967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.050834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9603b5-5a96-44eb-911a-ade73bf211c4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.056906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.057577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.057863 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.058141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.059442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.061071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.071958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1277/2523] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:59.076994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.087714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d3a3-b92f-4c6d-a127-d749875a524b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.093005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.093658 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.093948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.094247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.094961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.096606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ace1-8076-413e-9a2e-fa6ff2d84d61 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.107786 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1247/2524] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:59.112070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.123021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb381-70ee-4675-ab9e-b4600e405c10 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.127663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.128551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.128871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.129150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] PATH_INFO: `/v3/projects/c976def9b4974f90bf1fbf0a60c55993/users/52f0d14428b447cb817a7f496d207721/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.145336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c976def9b4974f90bf1fbf0a60c55993, user_id=52f0d14428b447cb817a7f496d207721, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.148120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.164697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1278/2525] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:59 2022] PUT /identity/v3/projects/c976def9b4974f90bf1fbf0a60c55993/users/52f0d14428b447cb817a7f496d207721/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:59.176601 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.176899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.177278 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.205960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] MFA Rules not processed for user `52f0d14428b447cb817a7f496d207721`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:59.207143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:59.274291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1248/2526] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:59.279181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.288451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:59.345935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f1a0-7969-4798-87af-90dc44382c1c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.351784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.352449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.352729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.352995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.353698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.355657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.387027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1279/2527] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:53:59 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:53:59.391891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.403991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd344641-7488-49fc-acca-2c3114c1eb60 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.473950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.474545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.474831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.475134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.475852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.477787 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.503171 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1249/2528] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:53:59 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:53:59.506854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.518250 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1362cb-0ec1-4562-8a82-1f16dd5b91a9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.523396 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.524096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.524381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.524664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.525400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.527017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.539297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1280/2529] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:59.544178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.556437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5735f3-59a9-47e6-9eb1-5433a19c5d1a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.565392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.566431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.566799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.567173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] PATH_INFO: `/v3/projects/725d05959a5f459b93aac00324e343e6/users/e9a3e5e6e8974dbab0d01fb396d07a88/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.585130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725d05959a5f459b93aac00324e343e6, user_id=e9a3e5e6e8974dbab0d01fb396d07a88, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.587950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.603707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1250/2530] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:59 2022] PUT /identity/v3/projects/725d05959a5f459b93aac00324e343e6/users/e9a3e5e6e8974dbab0d01fb396d07a88/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:59.608464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.618724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758ff46-08b3-457c-be49-1f58ac728350 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.677387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.678321 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.678697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.679094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.680219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.682751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a337003-ab97-4a9f-938d-c2915baafaf4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.698540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1281/2531] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:53:59.703402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.714431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4404d71-fba3-49a5-bf68-e8aaac75e3f7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.719645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.720581 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.720867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.721145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] PATH_INFO: `/v3/projects/725d05959a5f459b93aac00324e343e6/users/e9a3e5e6e8974dbab0d01fb396d07a88/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.723458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725d05959a5f459b93aac00324e343e6, user_id=e9a3e5e6e8974dbab0d01fb396d07a88, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.726753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.738757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1251/2532] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:53:59 2022] PUT /identity/v3/projects/725d05959a5f459b93aac00324e343e6/users/e9a3e5e6e8974dbab0d01fb396d07a88/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:53:59.749313 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.749611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.749892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.778680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] MFA Rules not processed for user `e9a3e5e6e8974dbab0d01fb396d07a88`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:53:59.779955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:59.849959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1282/2533] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:53:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:53:59.861755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:53:59.863521 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:53:59.868898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:53:59.922604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba33cf3-a478-4aff-ae36-41a96f438e15 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:53:59.926363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:53:59.927006 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:53:59.927316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:53:59.927587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:53:59.936857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:53:59.938279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:53:59.948055 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1252/2534] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:53:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:00.026082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:00.031685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:00.039737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23661d-453c-48c9-9d08-b4a7ab563c7d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:00.044825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:00.045577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:00.045861 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:00.046143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:00.108624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:00.110245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3883fa3-2b4c-4880-9bc9-342dbc8110c3 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:00.122873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1283/2535] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:00 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:09.943925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=25 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:10.116110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=27 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:13.790090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:13.792477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:13.873812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7098-872e-4ffd-ad39-f237872e7558 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:13.878742 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:13.879649 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:13.879930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:13.880220 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] PATH_INFO: `/v3/users/a025691cdaaa4654b5e8a23532eec54a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:13.881405 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a025691cdaaa4654b5e8a23532eec54a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:13.883353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:13.939580 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a025691cdaaa4654b5e8a23532eec54a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:13.940252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a025691cdaaa4654b5e8a23532eec54a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:13.949079 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a025691cdaaa4654b5e8a23532eec54a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:13.958740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a025691cdaaa4654b5e8a23532eec54a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:13.973862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a025691cdaaa4654b5e8a23532eec54a'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:13.981004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1253/2536] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:13 2022] DELETE /identity/v3/users/a025691cdaaa4654b5e8a23532eec54a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:13.986193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:13.986703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:13.986891 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:14.005131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:14.006323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:14.071253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1284/2537] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:14.081306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:14.082956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:14.144333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e817e-00eb-416a-89e8-0a4ac738d5ad admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:14.147937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:14.148586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:14.148866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:14.149138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:14.159624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:14.161350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:14.173097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1254/2538] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:14.471478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:14.473632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:14.496063 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83451a-07a3-45be-80a8-8b3c7d6746ca None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:14.501644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:14.502605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:14.502893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:14.503213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] PATH_INFO: `/v3/projects/744ee1c94ffb4553a0bcf84cf433defa` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:14.504546 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744ee1c94ffb4553a0bcf84cf433defa)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:14.506507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:14.557813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744ee1c94ffb4553a0bcf84cf433defa', 'request_id': 'req-48474078-bc0e-48eb-887b-bf8fb3475c75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.566096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744ee1c94ffb4553a0bcf84cf433defa', 'request_id': 'req-48474078-bc0e-48eb-887b-bf8fb3475c75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.572638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744ee1c94ffb4553a0bcf84cf433defa', 'request_id': 'req-48474078-bc0e-48eb-887b-bf8fb3475c75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.584534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] The token cache is being invalidate because project 744ee1c94ffb4553a0bcf84cf433defa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:14.584825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.585481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.587103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1285/2539] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:14 2022] DELETE /identity/v3/projects/744ee1c94ffb4553a0bcf84cf433defa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:14.591925 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:14.606867 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273dc7a-b21a-494e-a001-de0e42b4ca32 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:14.634805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:14.679943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:14.680830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:14.681107 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:14.681379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] PATH_INFO: `/v3/users/1280b485b0454379a22ebd0ff7c5cea0` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:14.682583 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1280b485b0454379a22ebd0ff7c5cea0)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:14.684749 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:14.745961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1280b485b0454379a22ebd0ff7c5cea0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.746915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1280b485b0454379a22ebd0ff7c5cea0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.755932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1280b485b0454379a22ebd0ff7c5cea0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.766383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1280b485b0454379a22ebd0ff7c5cea0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.779469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b8728483-252b-4abf-8d99-b6a0690064ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1280b485b0454379a22ebd0ff7c5cea0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:14.786292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1255/2540] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:14 2022] DELETE /identity/v3/users/1280b485b0454379a22ebd0ff7c5cea0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:15.064465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.065975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:15.085776 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474078-bc0e-48eb-887b-bf8fb3475c75 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.148021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.148774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.149051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.149306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] PATH_INFO: `/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.150601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d80873b0c5e45f487b5f93d3b9ab0a5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.152757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.185604 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:15.203291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d80873b0c5e45f487b5f93d3b9ab0a5', 'request_id': 'req-7b2fd130-940a-438b-997e-d23f1749c751'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:15.210284 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d80873b0c5e45f487b5f93d3b9ab0a5', 'request_id': 'req-7b2fd130-940a-438b-997e-d23f1749c751'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:15.216763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d80873b0c5e45f487b5f93d3b9ab0a5', 'request_id': 'req-7b2fd130-940a-438b-997e-d23f1749c751'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:15.227018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] The token cache is being invalidate because project 0d80873b0c5e45f487b5f93d3b9ab0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:15.227348 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:15.227963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7b2fd130-940a-438b-997e-d23f1749c751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:15.229549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1286/2541] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:15 2022] DELETE /identity/v3/projects/0d80873b0c5e45f487b5f93d3b9ab0a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:15.234837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.235663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.235828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.254830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:15.256131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:15.321567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1256/2542] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:15.329782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.330071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.330355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.351228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:15.352516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:15.361154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1287/2543] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:15.365587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.376355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb35bd-5428-4dbe-b641-b24f5ad28064 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.381257 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.381913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.382199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.382475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.383243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:15.383550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.385164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.397547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1257/2544] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:15.402136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.411800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d333-0cbc-4bae-b268-9b3a416b24ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.418485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.419300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.419689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.420055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.420838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.423009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.451608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1288/2545] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:15.455755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.466577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c170b-3ef3-4482-931f-de426c14d0ac admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.495240 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:15.536691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.537849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.538275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.538759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.543143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.543143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.567812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1258/2546] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/users => generated 279 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:54:15.572171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.583286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c7254-2c8b-4103-9b05-84ebaf124d0d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.587896 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.588557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.589050 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.589208 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.589851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.591780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.603131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1289/2547] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:15.607383 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.619531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2e6de-a0fd-4d56-9591-c3523ce2bea8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.625482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.626164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.626450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.626859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.627495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.629082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936752d8-f92f-4a80-a8b3-f6e5faba9a1b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.642864 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1259/2548] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:15.646584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.658449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db4d68-f47e-4ad0-88c0-0cfb0e5e7134 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.663336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.664280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.664569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.664852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] PATH_INFO: `/v3/projects/9e0fc6406c1e4ad3b6f71c4a3a4862be/users/3342317870e243499524ad724f00e2af/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.689409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0fc6406c1e4ad3b6f71c4a3a4862be, user_id=3342317870e243499524ad724f00e2af, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.692258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.708279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1290/2549] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:15 2022] PUT /identity/v3/projects/9e0fc6406c1e4ad3b6f71c4a3a4862be/users/3342317870e243499524ad724f00e2af/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:15.720108 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.720399 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.720682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.755927 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] MFA Rules not processed for user `3342317870e243499524ad724f00e2af`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:15.757475 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:15.832012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1260/2550] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:15.837144 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.901475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08ca4d-eeec-4eb9-b13a-28bff765670e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:15.906169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:15.906764 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:15.907045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:15.907382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:15.908060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:15.909966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:15.938138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1291/2551] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/projects => generated 355 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:15.942515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:15.954612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c4c8e-58ac-40f4-8eb8-96a1b225cd9f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.017191 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.017896 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.018175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.018448 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.019264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.021392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.043591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1261/2552] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:15 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:16.048089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.060845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7102643-43d8-475b-b49d-c0111ddfb311 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.067226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.067992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.068230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.068564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.069320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.071005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.083340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1292/2553] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:16.087891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.100643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38f40f-30a8-4027-8456-32a1c7491850 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.106358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.107406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.107791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.108161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] PATH_INFO: `/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab/users/3719293bfeb44c01a38d7e9bba718d37/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.124050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5e5d01d6f146aa9b5c28f051871eab, user_id=3719293bfeb44c01a38d7e9bba718d37, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.126906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.142003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1262/2554] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:16 2022] PUT /identity/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab/users/3719293bfeb44c01a38d7e9bba718d37/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:16.146125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.156692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05398d-34f5-4b3d-b04b-53abe2e5a211 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.178126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:16.220228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.220903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.221191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.221465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.222186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.223833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa88e0-cbc4-460e-bfdc-412516cefbd7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.235269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1293/2555] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:54:16.239950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.245995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:16.252736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2cd2e-bb99-4b7d-92ad-338f9aad5be1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.258002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.258985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.259393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.259715 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] PATH_INFO: `/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab/users/3719293bfeb44c01a38d7e9bba718d37/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.261887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5e5d01d6f146aa9b5c28f051871eab, user_id=3719293bfeb44c01a38d7e9bba718d37, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.265075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.276442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1263/2556] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:16 2022] PUT /identity/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab/users/3719293bfeb44c01a38d7e9bba718d37/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:16.287184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.287527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.287901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.316455 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] MFA Rules not processed for user `3719293bfeb44c01a38d7e9bba718d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:16.317702 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:16.390222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1294/2557] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:16 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:16.401895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.403712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:16.462869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ba84-39ec-4b62-9ab1-ab799a766386 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.467726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.468377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.468659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.468934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.478338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.479893 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.489454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1264/2558] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:16 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:16.569153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:16.581339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88d46d-de53-44f0-9127-57c907adcfe5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:16.585417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:16.586117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:16.586467 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:16.586829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:16.654624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:16.656190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546bc79-5ba1-4db7-a949-79d75b8b0f49 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:16.665428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1295/2559] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:16.999356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:17.000930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:17.072945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539c0d6-53c8-4405-ab82-bb8b4b42ef2f service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:17.079272 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:17.080167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:17.080440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:17.080739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] PATH_INFO: `/v3/users/52f0d14428b447cb817a7f496d207721` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:17.082105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f0d14428b447cb817a7f496d207721)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:17.084159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:17.136500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f0d14428b447cb817a7f496d207721'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.137174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f0d14428b447cb817a7f496d207721'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.144807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f0d14428b447cb817a7f496d207721'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.152616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f0d14428b447cb817a7f496d207721'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.165989 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f0d14428b447cb817a7f496d207721'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.173173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1265/2560] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:16 2022] DELETE /identity/v3/users/52f0d14428b447cb817a7f496d207721 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:17.178245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:17.179011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:17.179252 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:17.199276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:17.200516 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:17.244562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:17.265160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1296/2561] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:17.275341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:17.277250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:17.292369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:17.340501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a027a-802f-4f8d-a177-f5f09e22b38e admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:17.344211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:17.344866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:17.345147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:17.345418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:17.354983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:17.356460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:17.366201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1266/2562] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:17.640166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:17.641789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:17.657912 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6ad51-bb8f-4de1-b42f-5a716b0fef06 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:17.665707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:17.668248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:17.668760 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:17.669253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] PATH_INFO: `/v3/projects/c976def9b4974f90bf1fbf0a60c55993` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:17.671289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c976def9b4974f90bf1fbf0a60c55993)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:17.674257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:17.725521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c976def9b4974f90bf1fbf0a60c55993', 'request_id': 'req-d677c51d-5a4d-4fad-8f55-454a8a54213c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.733821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c976def9b4974f90bf1fbf0a60c55993', 'request_id': 'req-d677c51d-5a4d-4fad-8f55-454a8a54213c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.740831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c976def9b4974f90bf1fbf0a60c55993', 'request_id': 'req-d677c51d-5a4d-4fad-8f55-454a8a54213c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.752512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] The token cache is being invalidate because project c976def9b4974f90bf1fbf0a60c55993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:17.752742 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.753413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.755027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1297/2563] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:17 2022] DELETE /identity/v3/projects/c976def9b4974f90bf1fbf0a60c55993 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:17.759515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:17.770342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bbc24-2862-4a74-89b5-a301ce948657 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:17.834339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:17.835295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:17.835564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:17.835841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] PATH_INFO: `/v3/users/e9a3e5e6e8974dbab0d01fb396d07a88` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:17.837074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a3e5e6e8974dbab0d01fb396d07a88)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:17.839986 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:17.899043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a3e5e6e8974dbab0d01fb396d07a88'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.899863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a3e5e6e8974dbab0d01fb396d07a88'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.910008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a3e5e6e8974dbab0d01fb396d07a88'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.921260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a3e5e6e8974dbab0d01fb396d07a88'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.935822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-5ae70b57-f599-4053-b117-c46ac0c194e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a3e5e6e8974dbab0d01fb396d07a88'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:17.943098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1267/2564] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:17 2022] DELETE /identity/v3/users/e9a3e5e6e8974dbab0d01fb396d07a88 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:18.228336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.229974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:18.243868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677c51d-5a4d-4fad-8f55-454a8a54213c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.320351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.321272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.321555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.321805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] PATH_INFO: `/v3/projects/725d05959a5f459b93aac00324e343e6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.323199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=725d05959a5f459b93aac00324e343e6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.325108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.375184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725d05959a5f459b93aac00324e343e6', 'request_id': 'req-d14737ca-dd17-43ad-af7b-934035562b71'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:18.382708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725d05959a5f459b93aac00324e343e6', 'request_id': 'req-d14737ca-dd17-43ad-af7b-934035562b71'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:18.389581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725d05959a5f459b93aac00324e343e6', 'request_id': 'req-d14737ca-dd17-43ad-af7b-934035562b71'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:18.396055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:18.401355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] The token cache is being invalidate because project 725d05959a5f459b93aac00324e343e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:18.401771 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:18.402557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d14737ca-dd17-43ad-af7b-934035562b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:18.404378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1298/2565] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:18 2022] DELETE /identity/v3/projects/725d05959a5f459b93aac00324e343e6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:18.409594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.410114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.410233 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.428092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:18.429301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:18.460402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:18.501698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1268/2566] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:18.509676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.509998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.510279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.537914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:18.539150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:18.548698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1299/2567] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:18.553288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.563949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a6031-503a-4245-a7e2-2b441d029926 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.569471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.570125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.570410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.570690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.571459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:18.571778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.573469 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.586251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1269/2568] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:18.590622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.602412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135d22a-23b6-4c82-a84c-413f0686ed29 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.608724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.609384 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.609656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.609932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.610626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.612722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.642309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1300/2569] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:18.646663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.658156 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786675-fdd4-4832-9084-85af9b9b85cf admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.728111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.729049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.729370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.729703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.730637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.732684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.754451 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1270/2570] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:18 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:18.758700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.770425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a85302-755a-4449-a19b-32863a627191 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.775401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.776064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.776756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.776756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.777443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.779202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.789951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1301/2571] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:18.794731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.809813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef180-ea6c-437f-a218-4966a7f2bbc6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.815859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87552943-3667-481f-af06-9ba128918cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.816526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552943-3667-481f-af06-9ba128918cab admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.816813 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552943-3667-481f-af06-9ba128918cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.817092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87552943-3667-481f-af06-9ba128918cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.817831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87552943-3667-481f-af06-9ba128918cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.819481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87552943-3667-481f-af06-9ba128918cab admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.829994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1271/2572] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:18.834417 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:18.845715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7751941-cbf3-43e9-a2e2-0ed9067b3a65 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:18.851765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:18.852654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.852935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.853217 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] PATH_INFO: `/v3/projects/4369171d8bc1428f85eb36c64027e86c/users/bb1392c1e79d4a468e5408aefc7d5298/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.870653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4369171d8bc1428f85eb36c64027e86c, user_id=bb1392c1e79d4a468e5408aefc7d5298, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:18.873365 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:18.891112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1302/2573] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:18 2022] PUT /identity/v3/projects/4369171d8bc1428f85eb36c64027e86c/users/bb1392c1e79d4a468e5408aefc7d5298/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:18.901712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:18.902003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:18.902309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:18.930371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] MFA Rules not processed for user `bb1392c1e79d4a468e5408aefc7d5298`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:18.931606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:19.007130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1272/2574] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:19.011991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.076938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3632e01-9029-4b5c-8591-a51538168df8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.082368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.083016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.083326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.083618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.084330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.086182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.112754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1303/2575] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:19 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:19.116901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.127574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efee3-abde-45bb-bf3b-6ddca921a0a2 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.154968 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:19.189663 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.190341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.190627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.190908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.191660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.193743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.217243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1273/2576] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:19 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:19.221443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.232314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641769-d734-41b5-93b8-3f907df6f43b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.236993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.237734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.238080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.238426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.239193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.240826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.251859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1304/2577] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:19.256066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.266696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e02eb-182d-4ca0-83db-bc7323fc0601 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.271803 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.272979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.272979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.273235 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] PATH_INFO: `/v3/projects/f9f9bbe61a654204bebddcbd85190b52/users/fe578ddfb8344e2ea24cc2058e21ab28/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.291221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f9bbe61a654204bebddcbd85190b52, user_id=fe578ddfb8344e2ea24cc2058e21ab28, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.294367 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.313010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.320147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1274/2578] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:19 2022] PUT /identity/v3/projects/f9f9bbe61a654204bebddcbd85190b52/users/fe578ddfb8344e2ea24cc2058e21ab28/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:19.335181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04260020-7a4c-4742-a669-226d2d594484 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.401804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.402661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.402948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.403269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.404257 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.405880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11385af-5690-4ef5-bd21-976efcf87a0b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.421327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1305/2579] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:54:19.422511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.433701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf30d2-3e80-4f5c-b356-d740c454b35a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.438460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.439435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.439741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.440015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] PATH_INFO: `/v3/projects/f9f9bbe61a654204bebddcbd85190b52/users/fe578ddfb8344e2ea24cc2058e21ab28/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.441965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f9bbe61a654204bebddcbd85190b52, user_id=fe578ddfb8344e2ea24cc2058e21ab28, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.444654 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.455541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1275/2580] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:19 2022] PUT /identity/v3/projects/f9f9bbe61a654204bebddcbd85190b52/users/fe578ddfb8344e2ea24cc2058e21ab28/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:19.467064 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.467369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.467655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.482353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:19.505525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] MFA Rules not processed for user `fe578ddfb8344e2ea24cc2058e21ab28`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:19.506785 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:19.584482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1306/2581] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:19.597443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:19.599456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:19.664072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5937e1-bd13-4437-adf2-6e808fba3622 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:19.668976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:19.669630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:19.669914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:19.670188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:19.680768 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:19.682202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:19.693180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1276/2582] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:20.012983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.023562 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353e03e-393c-4c3e-a947-9264a47a789d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.028290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe578ddfb8344e2ea24cc2058e21ab28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9f9bbe61a654204bebddcbd85190b52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.028953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.029243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.029524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.030270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:20.030589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:20.032675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:20.046939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1307/2583] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:20.051292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.060604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6bc50-0b5b-4999-9053-8894ee9b54aa service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.065178 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe578ddfb8344e2ea24cc2058e21ab28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9f9bbe61a654204bebddcbd85190b52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.065832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.066118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.066396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.067135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:20.069041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:20.096072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1277/2584] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:20 2022] POST /identity/v3/projects => generated 353 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:20.100256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.110550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3970fab-2a09-4ef0-9f34-44f18d0bc8ac tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.174378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe578ddfb8344e2ea24cc2058e21ab28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9f9bbe61a654204bebddcbd85190b52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.175263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.175648 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.176039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.177112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:20.177580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:20.180215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:20.193067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1308/2585] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:20.230859 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.244039 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd558349-0b8e-4e97-90b0-bd195e83a556 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.251688 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe578ddfb8344e2ea24cc2058e21ab28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9f9bbe61a654204bebddcbd85190b52', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.253231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.253744 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.254201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] PATH_INFO: `/v3/projects/5544de3c08474762a25d29e3711fea07` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.256138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5544de3c08474762a25d29e3711fea07)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:20.259303 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:20.323492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5544de3c08474762a25d29e3711fea07', 'request_id': 'req-b562da97-35af-4440-8aa9-944f28fad2de'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.328238 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:20.330040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5544de3c08474762a25d29e3711fea07', 'request_id': 'req-b562da97-35af-4440-8aa9-944f28fad2de'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.341051 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5544de3c08474762a25d29e3711fea07', 'request_id': 'req-b562da97-35af-4440-8aa9-944f28fad2de'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.353194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] The token cache is being invalidate because project 5544de3c08474762a25d29e3711fea07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:20.353488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.354283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b562da97-35af-4440-8aa9-944f28fad2de tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.356456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1278/2586] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:20 2022] DELETE /identity/v3/projects/5544de3c08474762a25d29e3711fea07 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:20.579577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.581670 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:20.655903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57810abd-d970-4920-a5c4-074bb1ae45d1 tempest-VolumeQuotaClassesTest-476510576 tempest-VolumeQuotaClassesTest-476510576-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.669090 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:20.721016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.721906 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.722190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.722464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] PATH_INFO: `/v3/users/bb1392c1e79d4a468e5408aefc7d5298` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.723730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1392c1e79d4a468e5408aefc7d5298)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:20.725672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:20.778291 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1392c1e79d4a468e5408aefc7d5298'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.779108 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1392c1e79d4a468e5408aefc7d5298'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.786005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1392c1e79d4a468e5408aefc7d5298'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.799329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1392c1e79d4a468e5408aefc7d5298'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.809545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1392c1e79d4a468e5408aefc7d5298'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:20.817361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1309/2587] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:20 2022] DELETE /identity/v3/users/bb1392c1e79d4a468e5408aefc7d5298 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:20.824832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.825664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.825851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:20.857572 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:20.858880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:20.920265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1279/2588] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:20.930842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:20.933505 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:20.997775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cbce8-52f0-4623-936f-b3038b5fe450 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:20.997984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:20.998665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:20.998948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:20.999247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:21.009125 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:21.010925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:21.021216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1310/2589] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:21.335150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:21.340491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:21.356015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e4118-79a1-4351-a1b0-05935f4979d4 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:21.361001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:21.361884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:21.362340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:21.362615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] PATH_INFO: `/v3/projects/4369171d8bc1428f85eb36c64027e86c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:21.364070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4369171d8bc1428f85eb36c64027e86c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:21.365976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:21.416822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4369171d8bc1428f85eb36c64027e86c', 'request_id': 'req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.423258 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4369171d8bc1428f85eb36c64027e86c', 'request_id': 'req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.432411 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4369171d8bc1428f85eb36c64027e86c', 'request_id': 'req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.444395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] The token cache is being invalidate because project 4369171d8bc1428f85eb36c64027e86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:21.444687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.445323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.446884 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1280/2590] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:21 2022] DELETE /identity/v3/projects/4369171d8bc1428f85eb36c64027e86c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:21.451282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:21.462113 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861f2a6-629a-4917-8c76-049d32c1b49c service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:21.504061 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:21.525265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:21.526176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:21.526484 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:21.526755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] PATH_INFO: `/v3/users/fe578ddfb8344e2ea24cc2058e21ab28` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:21.527994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe578ddfb8344e2ea24cc2058e21ab28)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:21.529911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:21.579470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe578ddfb8344e2ea24cc2058e21ab28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.580129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe578ddfb8344e2ea24cc2058e21ab28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.587664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe578ddfb8344e2ea24cc2058e21ab28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.601209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe578ddfb8344e2ea24cc2058e21ab28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.610273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-593fefef-251b-4edc-9566-c1a6bcc87a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe578ddfb8344e2ea24cc2058e21ab28'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:21.616409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1311/2591] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:21 2022] DELETE /identity/v3/users/fe578ddfb8344e2ea24cc2058e21ab28 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:21.895409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:21.896954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:21.910639 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2702a-f618-479d-8ed9-f2ea3b37e5e5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:21.971560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:21.972437 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:21.972723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:21.973005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] PATH_INFO: `/v3/projects/f9f9bbe61a654204bebddcbd85190b52` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:21.974206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f9bbe61a654204bebddcbd85190b52)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:21.976190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:21.987207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:22.032323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f9bbe61a654204bebddcbd85190b52', 'request_id': 'req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:22.055617 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f9bbe61a654204bebddcbd85190b52', 'request_id': 'req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:22.064892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f9bbe61a654204bebddcbd85190b52', 'request_id': 'req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:22.078131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] The token cache is being invalidate because project f9f9bbe61a654204bebddcbd85190b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:22.078453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:22.079252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-688fee7e-2bc0-42cf-b2ea-dfc74fcd18c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:22.080877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1281/2592] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:21 2022] DELETE /identity/v3/projects/f9f9bbe61a654204bebddcbd85190b52 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:22.085925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.086213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.086489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.104541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:22.105750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:22.175446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1312/2593] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:54:22.183253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.183554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.183831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.206158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:22.207470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:22.216391 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1282/2594] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:22.220824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.232501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad3c5-973b-4cfa-85bd-1473e8540bec None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.239420 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.240422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.240710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.241009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.242297 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:22.242735 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.245073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.262350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1313/2595] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:22.269210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.283299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79ea1a-518a-458a-8ec5-97aebb00aa48 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.290395 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.291074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.291370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.291680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.292404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.294311 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.322386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1283/2596] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:22.326817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.339261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76fa18-3fdd-4f8e-aace-5de4cbbcdf4d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.387254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:22.403270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.403947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.404230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.404506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.405199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.407145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.429339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1314/2597] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:22.433537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.444123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87515496-8b76-4ab6-8e13-1b6e53b495e8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.449120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.449781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.450071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.450349 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.451111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.452726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.463403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1284/2598] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:22.467596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.477538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918be675-015f-43ab-b479-845963c0a3de admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.482421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.483103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.483388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.483665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.484386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.485971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb47d08-1131-48ca-bc6c-605efec10e23 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.496632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1315/2599] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:22.501190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.518158 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78a906-c4d4-426a-9f56-40617f980c48 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.526374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.527402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.527664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.527992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] PATH_INFO: `/v3/projects/95eda15649354775a915eb6c2ea11269/users/8e899cc6bddd42a09ebe8adfbca9bd83/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.548149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95eda15649354775a915eb6c2ea11269, user_id=8e899cc6bddd42a09ebe8adfbca9bd83, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.548149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.562850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1285/2600] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:22 2022] PUT /identity/v3/projects/95eda15649354775a915eb6c2ea11269/users/8e899cc6bddd42a09ebe8adfbca9bd83/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:22.573751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.574044 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.574326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.601828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] MFA Rules not processed for user `8e899cc6bddd42a09ebe8adfbca9bd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:22.603107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:22.678095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1316/2601] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:22.684086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.754882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54226e-79c8-4606-8794-5db73da85921 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.760092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.760941 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.761322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.761611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.762321 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.764597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.793208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1286/2602] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:22.797333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.809427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519a5a6-c0b6-4002-b6f6-d428b1ddf40b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.866596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.867314 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.867602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.867878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.868560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.870744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.894940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1317/2603] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:22 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:22.898931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.910133 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a2f8-6716-435b-bec6-77991b8ec230 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.916860 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.917877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.918162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.918436 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.919212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.920841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.932066 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1287/2604] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:22.936370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.947089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e8ff9-929f-42f6-9620-535d8db3b7da admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.953173 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.953836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.954128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.954386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.955237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:22.956837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1c3e1-a87f-45fa-b284-91295d4a9820 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:22.967523 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1318/2605] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:22.971998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:22.983262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b213e7-9476-455f-b5a0-18d54aec333f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:22.989150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:22.990076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:22.990368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:22.990645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] PATH_INFO: `/v3/projects/f99dd216c44341169ac7d4eebfc8d822/users/3c2eb9be536d472fb489019e5ba8b3a2/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:22.994050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:23.008904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99dd216c44341169ac7d4eebfc8d822, user_id=3c2eb9be536d472fb489019e5ba8b3a2, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.012305 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.027146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1288/2606] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:22 2022] PUT /identity/v3/projects/f99dd216c44341169ac7d4eebfc8d822/users/3c2eb9be536d472fb489019e5ba8b3a2/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:23.038695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.038991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.039305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.066552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] MFA Rules not processed for user `3c2eb9be536d472fb489019e5ba8b3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:23.067732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:23.088667 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:23.149186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1319/2607] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:23.154441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.219484 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5722c60-cca8-4400-92fd-3698711f8672 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.224695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.225396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.225680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.225957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.226644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.228579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.256068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1289/2608] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:23 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:23.260192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.275431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958322e-0927-4c65-a0ed-30cd24e98902 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.335460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.336543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.336897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.337180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.337894 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.343888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.360834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1320/2609] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:23 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:54:23.365481 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.376712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acdd8-6909-41e6-b61c-49a768efcccc admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.382728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.383652 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.383938 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.384221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.384993 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.387294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.404556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1290/2610] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:23.408688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.418901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590be3a-ab10-4f63-8662-4c384c52f94d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.424591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.425585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.425866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.426141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] PATH_INFO: `/v3/projects/714e038bcce24e02a0b73a6192234326/users/f5c864c9a73b484884e81648acaf80bd/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.442077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=714e038bcce24e02a0b73a6192234326, user_id=f5c864c9a73b484884e81648acaf80bd, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.444812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.458564 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1321/2611] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:23 2022] PUT /identity/v3/projects/714e038bcce24e02a0b73a6192234326/users/f5c864c9a73b484884e81648acaf80bd/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:23.462631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.473564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838f7e9-7c93-41e8-ae27-57848c0e33c7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.538426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.539455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.539735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.540165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.540998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.543526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995956e9-de6b-446b-b424-e1f8f377449d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.557770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1291/2612] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:23.561845 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.573752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cb4b7-e316-4606-9716-4b9d9b1dc18f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.579559 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.580453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.580784 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.580999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] PATH_INFO: `/v3/projects/714e038bcce24e02a0b73a6192234326/users/f5c864c9a73b484884e81648acaf80bd/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.583899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714e038bcce24e02a0b73a6192234326, user_id=f5c864c9a73b484884e81648acaf80bd, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.586482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.599538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1322/2613] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:23 2022] PUT /identity/v3/projects/714e038bcce24e02a0b73a6192234326/users/f5c864c9a73b484884e81648acaf80bd/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:23.609997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.610291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.610574 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.637046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] MFA Rules not processed for user `f5c864c9a73b484884e81648acaf80bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:23.638267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:23.685034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:23.713980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1292/2614] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:23.725974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.728220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:23.810624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45390-ed42-4a8e-9561-a50f6f9a5039 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.814706 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.815387 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.815681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.815963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.825920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.827402 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.837112 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1323/2615] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:23.878392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:23.879961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:23.893957 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef850c5-e67c-4385-ad12-1e3aa2d7c695 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:23.898049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:23.898703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:23.898992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:23.899618 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:23.962315 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:23.964384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cdf1-7f01-4157-9d4d-87f89f174c1c service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:23.973915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1293/2616] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:26.909240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:26.928540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:26.975272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad14107-dbbb-4390-b462-376b27aac011 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:26.979980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:26.980832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:26.981166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:26.981426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] PATH_INFO: `/v3/users/8e899cc6bddd42a09ebe8adfbca9bd83` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:26.982897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e899cc6bddd42a09ebe8adfbca9bd83)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:26.984856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:27.037741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e899cc6bddd42a09ebe8adfbca9bd83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.038395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e899cc6bddd42a09ebe8adfbca9bd83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.045565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e899cc6bddd42a09ebe8adfbca9bd83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.059558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e899cc6bddd42a09ebe8adfbca9bd83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.068687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e899cc6bddd42a09ebe8adfbca9bd83'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.075374 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1324/2617] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:26 2022] DELETE /identity/v3/users/8e899cc6bddd42a09ebe8adfbca9bd83 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:27.079877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:27.080397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:27.080498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:27.097917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:27.099281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:27.160541 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1294/2618] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:27.170883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:27.172824 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:27.239177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24301908-b762-4ef3-ac88-8545dfec991f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:27.243410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:27.244077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:27.244364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:27.245304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:27.255857 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:27.258251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:27.269752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1325/2619] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:27.549536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:27.550982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:27.568614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09586e85-b45d-4393-af02-a90d628d9b37 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:27.574457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:27.575263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:27.575549 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:27.575826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] PATH_INFO: `/v3/projects/95eda15649354775a915eb6c2ea11269` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:27.578194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95eda15649354775a915eb6c2ea11269)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:27.580377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:27.631731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95eda15649354775a915eb6c2ea11269', 'request_id': 'req-55664bbd-321a-4361-b027-8f81d3704a05'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.637314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95eda15649354775a915eb6c2ea11269', 'request_id': 'req-55664bbd-321a-4361-b027-8f81d3704a05'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.646228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95eda15649354775a915eb6c2ea11269', 'request_id': 'req-55664bbd-321a-4361-b027-8f81d3704a05'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.657487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] The token cache is being invalidate because project 95eda15649354775a915eb6c2ea11269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:27.657774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.658397 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.659966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1295/2620] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:27 2022] DELETE /identity/v3/projects/95eda15649354775a915eb6c2ea11269 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:27.664167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:27.674358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2adcf30-2300-4739-a078-74c885ec6beb service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:27.714521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:27.739639 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:27.741118 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:27.741611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:27.741978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] PATH_INFO: `/v3/users/3c2eb9be536d472fb489019e5ba8b3a2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:27.743870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2eb9be536d472fb489019e5ba8b3a2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:27.746237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:27.796901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2eb9be536d472fb489019e5ba8b3a2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.797737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2eb9be536d472fb489019e5ba8b3a2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.805480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2eb9be536d472fb489019e5ba8b3a2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.819379 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2eb9be536d472fb489019e5ba8b3a2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.829158 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2eb9be536d472fb489019e5ba8b3a2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:27.836596 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1326/2621] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:27 2022] DELETE /identity/v3/users/3c2eb9be536d472fb489019e5ba8b3a2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:28.142695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:28.144460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:28.155069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:28.162210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664bbd-321a-4361-b027-8f81d3704a05 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:28.231616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:28.232487 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:28.232766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:28.233037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] PATH_INFO: `/v3/projects/f99dd216c44341169ac7d4eebfc8d822` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:28.234247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99dd216c44341169ac7d4eebfc8d822)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:28.236197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:28.293671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99dd216c44341169ac7d4eebfc8d822', 'request_id': 'req-540a7b61-9053-4b77-a4ca-11b301f32a40'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.299626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99dd216c44341169ac7d4eebfc8d822', 'request_id': 'req-540a7b61-9053-4b77-a4ca-11b301f32a40'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.308901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99dd216c44341169ac7d4eebfc8d822', 'request_id': 'req-540a7b61-9053-4b77-a4ca-11b301f32a40'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.320200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] The token cache is being invalidate because project f99dd216c44341169ac7d4eebfc8d822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:28.320483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.321144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.323239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1296/2622] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:28 2022] DELETE /identity/v3/projects/f99dd216c44341169ac7d4eebfc8d822 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:28.328102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:28.330170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:28.349478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409195-b588-44ff-84e6-4b91326e044a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:28.411319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:28.412120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:28.412410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:28.412699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] PATH_INFO: `/v3/users/f5c864c9a73b484884e81648acaf80bd` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:28.413899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c864c9a73b484884e81648acaf80bd)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:28.415869 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:28.469247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c864c9a73b484884e81648acaf80bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.470019 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c864c9a73b484884e81648acaf80bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.476868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c864c9a73b484884e81648acaf80bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.481222 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:28.490272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c864c9a73b484884e81648acaf80bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.499762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3935feba-1d47-498c-9aa8-bec92b531e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c864c9a73b484884e81648acaf80bd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.505754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1327/2623] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:28 2022] DELETE /identity/v3/users/f5c864c9a73b484884e81648acaf80bd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:28.800894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:28.802578 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:28.820163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7b61-9053-4b77-a4ca-11b301f32a40 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:28.881950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:28.882862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:28.883186 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:28.883492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] PATH_INFO: `/v3/projects/714e038bcce24e02a0b73a6192234326` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:28.884719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=714e038bcce24e02a0b73a6192234326)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:28.889310 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:28.930377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:28.943821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714e038bcce24e02a0b73a6192234326', 'request_id': 'req-8c4d9eee-4908-4065-9d52-a34faf533c2c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.949954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714e038bcce24e02a0b73a6192234326', 'request_id': 'req-8c4d9eee-4908-4065-9d52-a34faf533c2c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.958733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714e038bcce24e02a0b73a6192234326', 'request_id': 'req-8c4d9eee-4908-4065-9d52-a34faf533c2c'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.970075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] The token cache is being invalidate because project 714e038bcce24e02a0b73a6192234326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:28.970363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.971031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8c4d9eee-4908-4065-9d52-a34faf533c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:28.972660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1297/2624] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:28 2022] DELETE /identity/v3/projects/714e038bcce24e02a0b73a6192234326 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:28.978749 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:28.979170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:28.979459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:28.998689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:29.000011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:29.062987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1328/2625] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:29.075038 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.075381 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.075664 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.095180 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:29.096403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:29.105519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1298/2626] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:29.110045 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.120775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84f77-adc6-459d-9902-8ecfd20e87f5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.127392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.128181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.128465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.128805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.129724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:29.130038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.131718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.144659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1329/2627] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:29.149460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.159387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cbf98-55f0-4f0a-ad29-8be8e195bc25 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.165009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.165660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.165939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.166208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.166914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.168908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.199861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1299/2628] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:29.205557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.216433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bea799-4112-4c5c-b594-230eb533d0e0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.274323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.274994 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.275315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.275594 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.276342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.278399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.297904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1330/2629] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:29.302498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.314063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a7cf-e4ab-49a8-9047-0b09dcabd1c0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.319616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.320359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.320630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.320901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.321629 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.323497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.334880 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1300/2630] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:29.339363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.349992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3794-3cd9-46f4-827c-043eafd9c9e1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.354752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.355443 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.355731 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.356012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.357584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.359243 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7ecd1-76c2-4563-b399-4f21cedfb82d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.369352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1331/2631] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:29.374119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.384847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceace0e-b29b-4106-99eb-306f7b3ba448 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.391263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.392124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.392441 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.392718 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] PATH_INFO: `/v3/projects/35f3f4d6d4b2494eb9b97ad9b90ede8c/users/2b47675f3b764398b423bddf561e9c88/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.409260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f3f4d6d4b2494eb9b97ad9b90ede8c, user_id=2b47675f3b764398b423bddf561e9c88, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.411998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.428472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1301/2632] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:29 2022] PUT /identity/v3/projects/35f3f4d6d4b2494eb9b97ad9b90ede8c/users/2b47675f3b764398b423bddf561e9c88/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:29.441398 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.441693 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.441963 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.470450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] MFA Rules not processed for user `2b47675f3b764398b423bddf561e9c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:29.471623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:29.507020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:29.543508 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1332/2633] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:29.548297 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.632822 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f679e05-e75e-4d84-b160-e197511f1645 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.639372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.640045 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.640329 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.640606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.641302 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.643251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.672838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1302/2634] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:29.678575 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.690931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5b2c-1aa9-4f38-b8c3-757d5f5f2701 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.750657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.751272 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.751556 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.751830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.752525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.754448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.774954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1333/2635] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:29 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:54:29.779125 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.794101 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24127f06-23dc-490c-af0f-4c514034cd4f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.799762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.800428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.800716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.800997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.801730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.803382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.815056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1303/2636] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:29.819307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.829647 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a126423-134a-48d2-b46b-c52c1497ce48 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.835070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.835956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.836236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.836528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] PATH_INFO: `/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802/users/dfb28c4b77124f47869a72651b18646e/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.853654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff10becaa96a4d53b76fe77fc5fdf802, user_id=dfb28c4b77124f47869a72651b18646e, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.856486 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.870391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1334/2637] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:29 2022] PUT /identity/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802/users/dfb28c4b77124f47869a72651b18646e/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:29.875495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.886102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd6e7e-1406-49d1-b793-611b0404cb79 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.923206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:29.954833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.955548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.955831 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.956216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.956945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:29.958573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336a789-a3c6-4b34-88bb-0c9c483f505e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:29.971815 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1304/2638] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:29.976056 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:29.988763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e867c0d-0807-4bac-b04f-8945cc8131b3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:29.994096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:29.995065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:29.995357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:29.995723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] PATH_INFO: `/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802/users/dfb28c4b77124f47869a72651b18646e/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:29.997650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff10becaa96a4d53b76fe77fc5fdf802, user_id=dfb28c4b77124f47869a72651b18646e, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:30.000382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:30.011663 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1335/2639] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:29 2022] PUT /identity/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802/users/dfb28c4b77124f47869a72651b18646e/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:30.022229 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:30.022518 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:30.022812 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:30.050455 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] MFA Rules not processed for user `dfb28c4b77124f47869a72651b18646e`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:30.051727 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:30.127169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1305/2640] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:30.139073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:30.141041 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:30.202898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1027d-1611-4b67-8ca4-5a1ff69f0f32 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:30.207018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:30.207711 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:30.208001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:30.208282 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:30.251084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:30.272202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:30.273653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8996c5-8c79-4e6f-8b21-8d392b0b4a9c service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:30.283423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1336/2641] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:31.635124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:31.636851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:31.709416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42783-41d6-4ceb-a61f-def2bcb8e055 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:31.716569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:31.717890 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:31.718192 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:31.718545 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] PATH_INFO: `/v3/users/3342317870e243499524ad724f00e2af` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:31.720109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3342317870e243499524ad724f00e2af)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:31.723074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:31.774226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3342317870e243499524ad724f00e2af'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:31.774877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3342317870e243499524ad724f00e2af'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:31.782933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3342317870e243499524ad724f00e2af'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:31.784978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:31.792478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3342317870e243499524ad724f00e2af'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:31.804735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3342317870e243499524ad724f00e2af'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:31.811202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1306/2642] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:31 2022] DELETE /identity/v3/users/3342317870e243499524ad724f00e2af => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:31.815915 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:31.816411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:31.816411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:31.834308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:31.835573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:31.899068 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1337/2643] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:54:31.909316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:31.911138 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:31.969449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102c19f-a108-45c9-87f0-2365f28d9193 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:31.973110 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:31.973760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:31.974046 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:31.974346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:31.984851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:31.986279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:31.996330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1307/2644] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:32.287367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:32.288904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:32.301914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357ae75-8cf5-4a63-8baf-9ef52a8eb034 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:32.306843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:32.307856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:32.308186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:32.308464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] PATH_INFO: `/v3/projects/9e0fc6406c1e4ad3b6f71c4a3a4862be` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:32.309715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0fc6406c1e4ad3b6f71c4a3a4862be)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:32.311687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:32.362643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0fc6406c1e4ad3b6f71c4a3a4862be', 'request_id': 'req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.372033 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0fc6406c1e4ad3b6f71c4a3a4862be', 'request_id': 'req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.378999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0fc6406c1e4ad3b6f71c4a3a4862be', 'request_id': 'req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.394848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] The token cache is being invalidate because project 9e0fc6406c1e4ad3b6f71c4a3a4862be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:32.395181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.396170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.398563 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1338/2645] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:32 2022] DELETE /identity/v3/projects/9e0fc6406c1e4ad3b6f71c4a3a4862be => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:32.402684 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:32.412538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a94d5e-5b7c-4f1a-aeb2-25478b6015da service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:32.471707 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:32.472514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:32.472790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:32.473061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] PATH_INFO: `/v3/users/3719293bfeb44c01a38d7e9bba718d37` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:32.474270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3719293bfeb44c01a38d7e9bba718d37)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:32.476232 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:32.523378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3719293bfeb44c01a38d7e9bba718d37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.524061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3719293bfeb44c01a38d7e9bba718d37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.532905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3719293bfeb44c01a38d7e9bba718d37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.540760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3719293bfeb44c01a38d7e9bba718d37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.554358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:32.556384 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-2c6d84a4-8dd2-494c-bde0-cb6acafbfed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3719293bfeb44c01a38d7e9bba718d37'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:32.566872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1308/2646] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:32 2022] DELETE /identity/v3/users/3719293bfeb44c01a38d7e9bba718d37 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:32.871674 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:32.873210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:32.887460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5a44c-abc6-42fd-9ee8-6a816e382e75 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:32.950804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:32.951704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:32.951992 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:32.952270 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] PATH_INFO: `/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:32.953511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5e5d01d6f146aa9b5c28f051871eab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:32.955494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:32.964002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:33.008388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5e5d01d6f146aa9b5c28f051871eab', 'request_id': 'req-7080a29f-1dc2-4805-bccb-f57559e9ed18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:33.018758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5e5d01d6f146aa9b5c28f051871eab', 'request_id': 'req-7080a29f-1dc2-4805-bccb-f57559e9ed18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:33.025879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5e5d01d6f146aa9b5c28f051871eab', 'request_id': 'req-7080a29f-1dc2-4805-bccb-f57559e9ed18'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:33.037600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] The token cache is being invalidate because project 0a5e5d01d6f146aa9b5c28f051871eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:33.037895 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:33.038635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7080a29f-1dc2-4805-bccb-f57559e9ed18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:33.040300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1339/2647] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:32 2022] DELETE /identity/v3/projects/0a5e5d01d6f146aa9b5c28f051871eab => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:33.046036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d080e0-b619-4895-a323-396727124964 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.046796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d080e0-b619-4895-a323-396727124964 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.046906 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d080e0-b619-4895-a323-396727124964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.065706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f2d080e0-b619-4895-a323-396727124964 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:33.066915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f2d080e0-b619-4895-a323-396727124964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:33.134282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1309/2648] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:33.142787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.143106 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.143403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.163316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:33.164536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:33.173391 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1340/2649] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:33.177686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d080e0-b619-4895-a323-396727124964 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.188607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d080e0-b619-4895-a323-396727124964 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.195012 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.195709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.195995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.196269 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.196970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:33.197389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.199118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.211882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1310/2650] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:54:33.217301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.227395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a8b3f-1298-46db-9013-caa18187bba3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.233325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.233991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.234275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.234550 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.235286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.237167 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.270077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1341/2651] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:33.274271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.284577 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2967c-98a4-422e-859c-a8b29a35de0f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.345794 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.346478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.346762 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.347141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.347824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.349738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.370028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1311/2652] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:54:33.374059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.385089 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3b124-3334-4ab8-9e79-4ce282a19182 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.393038 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.393703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.393995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.394273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.395018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.396697 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.409503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1342/2653] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:33.413368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.424313 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75023ff-b8b6-4751-84c1-bc8ebbfc75eb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.429551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.430312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.430602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.430974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.431805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.433732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de1916-e84f-4631-bb6e-94c1f96a7373 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.448703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1312/2654] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:33.450539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.462049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a54d5-a062-4dba-92f6-ab50dec9fa01 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.466787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.467770 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.468069 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.468327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] PATH_INFO: `/v3/projects/5f7d7ecbf222486c8446e34648a20d69/users/8d789fa67054410ab876eb72662ca76b/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.484094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7d7ecbf222486c8446e34648a20d69, user_id=8d789fa67054410ab876eb72662ca76b, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.488055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.502704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1343/2655] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:33 2022] PUT /identity/v3/projects/5f7d7ecbf222486c8446e34648a20d69/users/8d789fa67054410ab876eb72662ca76b/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:33.514477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.514774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.515307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.551631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] MFA Rules not processed for user `8d789fa67054410ab876eb72662ca76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:33.552900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:33.609533 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:33.628366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1313/2656] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:33.633581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.698615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c72e5e-304c-4c59-9c4a-d682a97beb96 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.703411 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.704078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.704361 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.704642 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.705355 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.707369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.733931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1344/2657] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:33.738026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.749613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6d471-8b52-4aa4-85b4-bda137ab5d03 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.806248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.806933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.807247 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.807567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.808334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.810304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.831477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1314/2658] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:33 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:33.835521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.846228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b166dbe-75d0-4d91-983f-98c3b5dd8dc0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.851638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.852304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.852591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.852872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.853605 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.855464 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.866427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1345/2659] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:33.870691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.880873 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bc537-7f9d-4e7e-a969-95e261c86033 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:33.886060 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:33.886944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:33.887260 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:33.887537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] PATH_INFO: `/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83/users/367f4c33f2094ea396cd8159cde8545b/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:33.903413 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afb0f7db2984500a4c7f89a25dcfc83, user_id=367f4c33f2094ea396cd8159cde8545b, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:33.906113 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:33.921603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1315/2660] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:33 2022] PUT /identity/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83/users/367f4c33f2094ea396cd8159cde8545b/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:33.925625 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:33.932435 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:33.937021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb346e4-0831-438c-987a-683882f89d34 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:34.001967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:34.002635 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:34.002921 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:34.003229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:34.003956 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:34.005574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7502d1c-ec18-4abc-8c38-8802dd922e37 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:34.015954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1346/2661] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:34.020404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:34.031624 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc30bf-70cf-45d9-8050-d17c8b51a0ae admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:34.037520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:34.038408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:34.038694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:34.038939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] PATH_INFO: `/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83/users/367f4c33f2094ea396cd8159cde8545b/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:34.041459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afb0f7db2984500a4c7f89a25dcfc83, user_id=367f4c33f2094ea396cd8159cde8545b, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:34.044159 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:34.055668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1316/2662] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:34 2022] PUT /identity/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83/users/367f4c33f2094ea396cd8159cde8545b/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:34.067088 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:34.067331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:34.067622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:34.097538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] MFA Rules not processed for user `367f4c33f2094ea396cd8159cde8545b`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:34.098775 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:34.175590 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1347/2663] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:54:34.187982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:34.189995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:34.251214 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea05d45-b4dc-4344-82cd-a258f705c95a admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:34.255287 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:34.255934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:34.256224 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:34.256505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:34.318879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:34.321017 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:34.325210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:34.333152 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1317/2664] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:34.702065 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:34.712198 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93d1e7-4dd8-47e7-a00b-b3e1dcb65064 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:34.716066 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:34.716708 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:34.716990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:34.717264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:34.719023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:34.790879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:34.792395 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:34.802124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1348/2665] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:54:42.090937 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:42.092483 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:42.105756 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c548b0-b609-40b3-979c-cbd4277395c8 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:42.110373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:42.111042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:42.111339 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:42.111614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:42.122018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:42.123511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c8094-9ac7-481a-a716-1930a385e94e service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:42.133182 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1318/2666] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:44.797888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:50.308569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:50.310901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:50.379329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214839b-3bcf-44c3-95ac-c848b0f8f18b service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:50.384239 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:50.385316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:50.385585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:50.385877 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] PATH_INFO: `/v3/users/2b47675f3b764398b423bddf561e9c88` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:50.387138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b47675f3b764398b423bddf561e9c88)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:50.389354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:50.437023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b47675f3b764398b423bddf561e9c88'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:50.437972 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b47675f3b764398b423bddf561e9c88'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:50.444993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b47675f3b764398b423bddf561e9c88'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:50.457948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b47675f3b764398b423bddf561e9c88'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:50.465957 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b47675f3b764398b423bddf561e9c88'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:50.473309 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1349/2667] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:50 2022] DELETE /identity/v3/users/2b47675f3b764398b423bddf561e9c88 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:50.481609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:50.482147 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:50.482338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:50.503673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:50.505536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:50.579648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1319/2668] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:50.590827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:50.592971 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:50.658866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7019e1c-db75-46bf-9b3d-4f83d67c2765 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:50.663595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:50.664248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:50.664529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:50.664799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:50.676177 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:50.677725 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:50.688978 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1350/2669] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:50.987901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:50.989459 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:51.002961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dee7a-10e2-4917-ae34-a0dc6aabab10 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:51.008753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:51.009721 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.010004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.010280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] PATH_INFO: `/v3/projects/35f3f4d6d4b2494eb9b97ad9b90ede8c` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.011555 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f3f4d6d4b2494eb9b97ad9b90ede8c)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:51.013780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:51.063378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f3f4d6d4b2494eb9b97ad9b90ede8c', 'request_id': 'req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.069856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f3f4d6d4b2494eb9b97ad9b90ede8c', 'request_id': 'req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.081198 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f3f4d6d4b2494eb9b97ad9b90ede8c', 'request_id': 'req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.093610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] The token cache is being invalidate because project 35f3f4d6d4b2494eb9b97ad9b90ede8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:51.093899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.094566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.096307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1320/2670] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:50 2022] DELETE /identity/v3/projects/35f3f4d6d4b2494eb9b97ad9b90ede8c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:51.100525 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:51.114181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64544b69-5d46-4b00-b4aa-fae5440aa9b9 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:51.141335 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:51.198656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:51.199561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.199883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.200155 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] PATH_INFO: `/v3/users/dfb28c4b77124f47869a72651b18646e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.201589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb28c4b77124f47869a72651b18646e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:51.203382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:51.257610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb28c4b77124f47869a72651b18646e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.258308 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb28c4b77124f47869a72651b18646e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.266524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb28c4b77124f47869a72651b18646e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.279910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb28c4b77124f47869a72651b18646e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.287989 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-61f867b5-1d84-4f20-88c1-223ba6600739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb28c4b77124f47869a72651b18646e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.294806 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1351/2671] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:51 2022] DELETE /identity/v3/users/dfb28c4b77124f47869a72651b18646e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:51.590281 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:51.592217 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:51.607027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c2fd7-84a8-4f08-b5ba-287a04dcac85 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:51.668402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:51.669282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.669571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.669853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] PATH_INFO: `/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.671069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff10becaa96a4d53b76fe77fc5fdf802)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:51.673008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:51.683745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:51.722280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff10becaa96a4d53b76fe77fc5fdf802', 'request_id': 'req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.728607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff10becaa96a4d53b76fe77fc5fdf802', 'request_id': 'req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.736877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff10becaa96a4d53b76fe77fc5fdf802', 'request_id': 'req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.749346 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] The token cache is being invalidate because project ff10becaa96a4d53b76fe77fc5fdf802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:51.749644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.750374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8fbd244-5a76-42bf-b10a-c51e47c13dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:51.752058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1321/2672] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:51 2022] DELETE /identity/v3/projects/ff10becaa96a4d53b76fe77fc5fdf802 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:51.758148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.758738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.758738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.778696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:51.779969 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:51.840179 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1352/2673] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:51.850171 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.850461 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.850716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.871540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:51.872772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:51.881680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1322/2674] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:51.886889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:51.897407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c805fe4-dfe5-4dc8-a71b-07307a77e91b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:51.903134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:51.903798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.904082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.904357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.905096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:51.905407 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:51.907094 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:51.919371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1353/2675] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:51.923946 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:51.934169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315c4c8-e4a0-4a40-b5eb-995ffa4c89b0 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:51.939883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:51.940590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:51.940856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:51.941161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:51.941858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:51.943827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:51.970667 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1323/2676] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:51 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:51.975003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:51.985631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07147347-0ace-4e05-8b37-12c0c0c2032c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:52.011936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:52.044698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:52.045376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.045664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.045944 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.046638 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:52.048618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:52.070927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1354/2677] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:51 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:52.075532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:52.087386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9e670-db6e-477a-a54a-2bd2e8aabee2 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:52.092954 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:52.093627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.093917 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.094223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.094943 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:52.096606 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:52.108300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1324/2678] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:52.112450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:52.123522 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97ca91-756e-46bb-aba6-b726bb825a2c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:52.129012 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:52.129681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.129970 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.130248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.130980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:52.132650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c4ec3-2902-471c-aa35-b8d7ef4d919c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:52.145734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1355/2679] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:52.150908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:52.161547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56530ffe-0d1d-4f81-b5dd-44d6de45b10f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:52.167793 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:52.168714 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.168977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.169282 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] PATH_INFO: `/v3/projects/23856cbe267842bdabfc5b4850b428b8/users/85d6fa485ec440f9880f3b48c8c89850/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.186241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23856cbe267842bdabfc5b4850b428b8, user_id=85d6fa485ec440f9880f3b48c8c89850, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:52.189782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:52.209208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1325/2680] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:52 2022] PUT /identity/v3/projects/23856cbe267842bdabfc5b4850b428b8/users/85d6fa485ec440f9880f3b48c8c89850/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:54:52.221052 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.221368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.221653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.248747 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] MFA Rules not processed for user `85d6fa485ec440f9880f3b48c8c89850`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:52.249892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:52.314247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1356/2681] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:52.326371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:52.328303 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:52.389511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f435f7-4bc3-41a1-a809-a65fe6fe8d03 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:52.393498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:52.394149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:52.394434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:52.394709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:52.403641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:52.405075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9110087-036c-41ff-9dca-35f133db32e1 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:52.413958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1326/2682] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:57.619031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:57.686678 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c46534-85e6-4cae-9218-9193e0bec5d7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:57.691948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:57.692928 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:57.693152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:57.693432 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] PATH_INFO: `/v3/users/85d6fa485ec440f9880f3b48c8c89850` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:57.696410 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d6fa485ec440f9880f3b48c8c89850)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:57.698320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:57.740704 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:57.747538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d6fa485ec440f9880f3b48c8c89850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:57.748750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d6fa485ec440f9880f3b48c8c89850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:57.756661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d6fa485ec440f9880f3b48c8c89850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:57.770681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d6fa485ec440f9880f3b48c8c89850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:57.785555 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d6fa485ec440f9880f3b48c8c89850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:57.793481 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1357/2683] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:54:57 2022] DELETE /identity/v3/users/85d6fa485ec440f9880f3b48c8c89850 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:57.798169 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:57.798668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:57.798783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:57.819323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:57.820558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:57.859883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:57.890300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1327/2684] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:57.900817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:57.902366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:57.966315 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3aa62-1064-4aab-918c-20cbff0a16e5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:57.970746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:57.971444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:57.971730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:57.972005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:57.981913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:57.983409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae0452-d1c1-4a95-aefe-d1f9ce5d6346 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:57.992924 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1358/2685] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:54:58.310505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.312161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:58.325030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34662-dd81-44fe-9960-b1812ae6c350 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.330550 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.331492 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.331778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.332049 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] PATH_INFO: `/v3/projects/23856cbe267842bdabfc5b4850b428b8` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.333470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23856cbe267842bdabfc5b4850b428b8)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.335342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.385308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23856cbe267842bdabfc5b4850b428b8', 'request_id': 'req-98787efe-470e-4191-bb42-5ca0b064f8e4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:58.391673 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23856cbe267842bdabfc5b4850b428b8', 'request_id': 'req-98787efe-470e-4191-bb42-5ca0b064f8e4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:58.401888 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23856cbe267842bdabfc5b4850b428b8', 'request_id': 'req-98787efe-470e-4191-bb42-5ca0b064f8e4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:58.413789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] The token cache is being invalidate because project 23856cbe267842bdabfc5b4850b428b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:54:58.414077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:58.414777 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-98787efe-470e-4191-bb42-5ca0b064f8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:54:58.416368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1328/2686] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:54:58 2022] DELETE /identity/v3/projects/23856cbe267842bdabfc5b4850b428b8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:58.421926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.422215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.422515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.441296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:58.442527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:58.519428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1359/2687] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:54:58.527463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.527760 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.528044 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.547858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:58.549075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:58.559920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1329/2688] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:58.564333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.574803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4aa9e7-32b5-44dc-aabf-dcbb42144fda None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.580135 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.580796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.581083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.581364 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.582109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:54:58.582426 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.584073 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.596854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1360/2689] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:54:58.601471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.612070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4d6b6-c12a-4f96-84af-8c8551c76ca9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.618163 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.619301 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.619569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.619869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.620570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.622495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.654988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1330/2690] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:54:58 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:58.656221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.666183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b1e4-255d-46d2-9147-c7a3e78d0adf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.715619 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:58.730967 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.731685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.731966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.732254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.732951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.734901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.759837 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1361/2691] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:54:58 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:54:58.765835 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.777950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a441be-2cbe-4612-9203-c463ff3a7988 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.784031 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.784661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.784933 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.785208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.785960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.789160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.800320 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1331/2692] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:58.805175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.816609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b60be-b7b5-40f1-99ba-a2d08a571055 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.822332 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.823010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.823331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.823608 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.824507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.825973 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ecce3-0270-4625-8808-bbe66e79550b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.836790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1362/2693] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:54:58.841872 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.852149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76bf62-ed8f-4b0e-9d62-e91c9992762c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:58.857726 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:58.858623 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.858909 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.859221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] PATH_INFO: `/v3/projects/0a55a4c28c4140958e1c078eaf57f9fd/users/690436e83b26412cb5c84f47202834f4/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.875351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a55a4c28c4140958e1c078eaf57f9fd, user_id=690436e83b26412cb5c84f47202834f4, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:58.878080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:58.892877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1332/2694] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:54:58 2022] PUT /identity/v3/projects/0a55a4c28c4140958e1c078eaf57f9fd/users/690436e83b26412cb5c84f47202834f4/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:54:58.898382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eae06-3426-4d8e-8fc6-2b4e3fda8214 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:58.898654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eae06-3426-4d8e-8fc6-2b4e3fda8214 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:58.898984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6eae06-3426-4d8e-8fc6-2b4e3fda8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:58.917193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cd6eae06-3426-4d8e-8fc6-2b4e3fda8214 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:54:58.918400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cd6eae06-3426-4d8e-8fc6-2b4e3fda8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:58.981470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1363/2695] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:54:58.991270 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:54:58.993400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:54:59.056200 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b9a36-d432-4edc-adbd-4139e7e1c0aa admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:54:59.060075 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:54:59.060732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:54:59.061015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:54:59.061289 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:54:59.070565 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:54:59.072063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:54:59.077164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:54:59.084129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1333/2696] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:03.390185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.390675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.390763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.457251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] MFA Rules not processed for user `690436e83b26412cb5c84f47202834f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:03.458501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:03.534680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1364/2697] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:03.535904 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:03.546128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66edf2-ebdd-4b3b-ad1b-9b9edfec01c0 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:03.551111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:03.551852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.552286 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.553061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.554534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:03.558208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:03.593252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1334/2698] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:55:03 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:55:03.597471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:03.609689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822cd89-97f8-40d2-ad29-ee4ec0147602 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:03.698615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:03.710597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:03.711581 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.712846 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.713209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.714203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:03.717552 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:03.748209 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1365/2699] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:55:03 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:03.752330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:03.766908 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f59e8-10b2-4328-9cb5-332648e3f505 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:03.773077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:03.773959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.774308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.774585 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.775457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:03.777176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:03.789496 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1335/2700] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:03.793703 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:03.809279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a570f-97f8-4030-9b11-43ddf776eb0c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:03.814814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:03.815991 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.816378 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.816793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] PATH_INFO: `/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e/users/bd0c5158f950442dabe9781948ea8fb2/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.836537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d522f3b82f84bd78aeeec19b2e49e5e, user_id=bd0c5158f950442dabe9781948ea8fb2, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:03.839888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:03.859441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1366/2701] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:03 2022] PUT /identity/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e/users/bd0c5158f950442dabe9781948ea8fb2/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:55:03.863323 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:03.882490 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938031-2d71-4b1c-b280-b634e7208daa admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:03.966333 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:03.967037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:03.967352 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:03.967660 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:03.968406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:03.970027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bc401-2a54-40f2-85cc-ad5291026b05 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:03.981564 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1336/2702] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:03.986031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:04.006334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3afef-b970-4490-8aad-b7b4cfa48d5b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:04.013333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:04.014371 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:04.014715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:04.015035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] PATH_INFO: `/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e/users/bd0c5158f950442dabe9781948ea8fb2/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:04.017251 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d522f3b82f84bd78aeeec19b2e49e5e, user_id=bd0c5158f950442dabe9781948ea8fb2, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:04.020274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:04.032821 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1367/2703] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:03 2022] PUT /identity/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e/users/bd0c5158f950442dabe9781948ea8fb2/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:08.601827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e2db-088d-4f0e-b35a-0be4e4b0646d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:08.602318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e2db-088d-4f0e-b35a-0be4e4b0646d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:08.602318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e2db-088d-4f0e-b35a-0be4e4b0646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:08.645376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ddd6e2db-088d-4f0e-b35a-0be4e4b0646d None None] MFA Rules not processed for user `bd0c5158f950442dabe9781948ea8fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:08.646607 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ddd6e2db-088d-4f0e-b35a-0be4e4b0646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:08.740284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1337/2704] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:08.756665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:08.759273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:08.856343 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d8c69-ec9d-4c11-8c46-292da57780d4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:08.859982 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ce6a5053f24cff8997908c6d7948a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:08.860641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:08.860926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:08.861203 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:08.924545 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:08.926013 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:08.938241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1368/2705] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:09.094985 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:09.095535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:09.095832 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:09.176265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:09.177604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:09.189530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:09.265889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1338/2706] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:55:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:09.327229 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:09.328798 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:09.380248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:09.416376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36539-5188-4608-b557-c9d253cff677 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:09.421640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:09.422310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:09.422591 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:09.422854 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:09.435736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:09.435736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e60d91-0a05-4872-bc15-4c817f8bef29 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:09.443304 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1369/2707] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:55:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:55:12.905621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:12.907912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:13.007854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9780-30c9-40a8-a4a7-0e7fc5a0355f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:13.014842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:13.015934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:13.016338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:13.016736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] PATH_INFO: `/v3/users/8d789fa67054410ab876eb72662ca76b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:13.018655 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d789fa67054410ab876eb72662ca76b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:13.021423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:13.083255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d789fa67054410ab876eb72662ca76b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.084002 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d789fa67054410ab876eb72662ca76b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.093520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d789fa67054410ab876eb72662ca76b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.102672 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d789fa67054410ab876eb72662ca76b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.130274 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d789fa67054410ab876eb72662ca76b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.141966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1339/2708] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:12 2022] DELETE /identity/v3/users/8d789fa67054410ab876eb72662ca76b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:13.146800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:13.147319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:13.147423 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:13.173829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:13.175504 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:13.284732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1370/2709] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:13.297154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:13.298788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:13.344458 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:13.403712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e13298-4e31-4635-8de5-47ae5407c86c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:13.408400 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:13.409146 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:13.409558 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:13.409855 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:13.420554 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:13.422149 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:13.447625 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1340/2710] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:13.835199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:13.837042 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:13.858109 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e0bb-5485-4a8b-b6e6-be5cf0d85b35 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:13.874858 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:13.875911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:13.876515 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:13.876871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] PATH_INFO: `/v3/projects/5f7d7ecbf222486c8446e34648a20d69` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:13.878393 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7d7ecbf222486c8446e34648a20d69)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:13.880512 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:13.955626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7d7ecbf222486c8446e34648a20d69', 'request_id': 'req-76e85355-ffef-4de2-8350-5919c198c5f0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.972362 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7d7ecbf222486c8446e34648a20d69', 'request_id': 'req-76e85355-ffef-4de2-8350-5919c198c5f0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.981075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7d7ecbf222486c8446e34648a20d69', 'request_id': 'req-76e85355-ffef-4de2-8350-5919c198c5f0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.994959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] The token cache is being invalidate because project 5f7d7ecbf222486c8446e34648a20d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:13.995489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.996256 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:13.997840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1371/2711] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:13 2022] DELETE /identity/v3/projects/5f7d7ecbf222486c8446e34648a20d69 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:14.003766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:14.015416 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613275e4-56bd-4040-8d4f-c4a4e48b0617 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:14.088593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:14.089945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:14.090419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:14.090879 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] PATH_INFO: `/v3/users/367f4c33f2094ea396cd8159cde8545b` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:14.092366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=367f4c33f2094ea396cd8159cde8545b)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:14.094498 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:14.163878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367f4c33f2094ea396cd8159cde8545b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.166965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367f4c33f2094ea396cd8159cde8545b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.182080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367f4c33f2094ea396cd8159cde8545b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.200474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367f4c33f2094ea396cd8159cde8545b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.216827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-00805208-fbf6-483e-9be9-fd59fe6ef5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367f4c33f2094ea396cd8159cde8545b'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.230650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1341/2712] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:14 2022] DELETE /identity/v3/users/367f4c33f2094ea396cd8159cde8545b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:14.593024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:14.599301 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:14.651349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e85355-ffef-4de2-8350-5919c198c5f0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:14.738454 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:14.760032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:14.760835 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:14.761473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:14.761473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] PATH_INFO: `/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:14.764479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afb0f7db2984500a4c7f89a25dcfc83)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:14.766431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:14.826627 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afb0f7db2984500a4c7f89a25dcfc83', 'request_id': 'req-bbee9aca-510e-479a-a17c-7738f52ae3a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.835276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afb0f7db2984500a4c7f89a25dcfc83', 'request_id': 'req-bbee9aca-510e-479a-a17c-7738f52ae3a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.845796 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afb0f7db2984500a4c7f89a25dcfc83', 'request_id': 'req-bbee9aca-510e-479a-a17c-7738f52ae3a9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.857249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] The token cache is being invalidate because project 7afb0f7db2984500a4c7f89a25dcfc83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:14.857666 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.858388 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bbee9aca-510e-479a-a17c-7738f52ae3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:14.860403 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1372/2713] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:14 2022] DELETE /identity/v3/projects/7afb0f7db2984500a4c7f89a25dcfc83 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:14.866175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:14.866603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:14.866887 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:14.900783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:14.902058 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:14.958431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:14.977676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1342/2714] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:14.986543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:14.986925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:14.987316 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.010254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:15.011434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:15.020449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1373/2715] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:15.025753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.036228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360beb-48b4-4e45-9dc7-03a45f0b834f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.042328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.043015 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.043560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.043696 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.044358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:55:15.044670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.046319 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.063743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1343/2716] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:55:15.064948 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.077329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f502bb-fd17-4bd1-890b-f398747e4eeb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.085951 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.087514 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.087815 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.088100 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.088829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.090810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.130224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1374/2717] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:55:15 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:15.131703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.143728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098514f-f7eb-4b62-9160-a0336bce81f0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.214735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.215593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.215956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.216312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.217076 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.219102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.249376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1344/2718] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:55:15 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:15.252188 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.263009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c7744-a9b9-4acb-908d-8a0ab8c2e9d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.271585 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.272392 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.272741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.273237 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.274149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.275905 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.287937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1375/2719] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:15.292071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.302934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123fdba-f5cb-469d-8a95-7322d125ecc4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.308816 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.309571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.309934 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.310279 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.311115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.313193 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711f1d-c3ac-4fa6-af8d-4d7226fd03c1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.331251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1345/2720] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:15.332677 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.346169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b80c-c85e-4303-a99f-def0a0ecc7df admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.351631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.352624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.352904 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.353176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] PATH_INFO: `/v3/projects/e3658c9591db45b48802ddebdd1deb39/users/7e6cae1e5a0f4c23951280814b9f6a13/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.372279 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3658c9591db45b48802ddebdd1deb39, user_id=7e6cae1e5a0f4c23951280814b9f6a13, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.374935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.390737 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1376/2721] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:15 2022] PUT /identity/v3/projects/e3658c9591db45b48802ddebdd1deb39/users/7e6cae1e5a0f4c23951280814b9f6a13/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:15.401454 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.402241 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.402810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.434018 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] MFA Rules not processed for user `7e6cae1e5a0f4c23951280814b9f6a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:15.435338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:15.526977 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1346/2722] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:15.528346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.613907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0abc52-5a24-487e-9a6b-563f84718ff7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.620883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.620883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.620883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.622722 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.626847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.628898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.652224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:15.662883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1377/2723] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:55:15 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:15.670086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.682361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c31bf-2879-496a-9311-5be87a73b7cb None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.794662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:15.833137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.834449 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.835042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.835645 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.837188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.845082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.899544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1347/2724] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:55:15 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:15.901018 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.918968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3687ea-dea2-41d1-a508-cd71be0ec5bf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.924449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.925867 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.926397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.926740 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.927640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.929589 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:15.947827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1378/2725] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:55:15.952433 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:15.964515 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e33a-1bdb-4ab5-94bc-04279a122ace admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:15.969524 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:15.970456 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:15.970745 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:15.971019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] PATH_INFO: `/v3/projects/a5f32cbfc0ef49f99420863c17ede421/users/29303317c8454fdf9f0e18824534e6a1/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:15.990478 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f32cbfc0ef49f99420863c17ede421, user_id=29303317c8454fdf9f0e18824534e6a1, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:15.997586 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.013900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1348/2726] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:15 2022] PUT /identity/v3/projects/a5f32cbfc0ef49f99420863c17ede421/users/29303317c8454fdf9f0e18824534e6a1/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:16.018487 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:16.039085 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c927a4e-8e28-4be2-815f-90e22ec4f763 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:16.109367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:16.110134 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.110530 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.110818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.111601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:16.113262 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1db1f-aa00-4146-925e-d55ed398992d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.127093 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1379/2727] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:16.128717 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:16.140361 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851774d-c3aa-4e73-963d-97334af52057 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:16.145501 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:16.146869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.146869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.147215 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] PATH_INFO: `/v3/projects/a5f32cbfc0ef49f99420863c17ede421/users/29303317c8454fdf9f0e18824534e6a1/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.149140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f32cbfc0ef49f99420863c17ede421, user_id=29303317c8454fdf9f0e18824534e6a1, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:16.151910 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.165116 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1349/2728] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:16 2022] PUT /identity/v3/projects/a5f32cbfc0ef49f99420863c17ede421/users/29303317c8454fdf9f0e18824534e6a1/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:55:16.179790 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.180081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.180366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.212260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] MFA Rules not processed for user `29303317c8454fdf9f0e18824534e6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:16.213611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:16.317373 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1380/2729] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:16.329881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:16.331892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:16.403928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9789-755d-4187-b313-ad041e410f53 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:16.407602 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:16.408328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.408679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.409023 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.421389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:16.423087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ec591-6502-4e9a-aef1-f64323d0fafc service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.439243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1350/2730] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:16.469818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:16.544632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c51c2-619a-4a30-8587-8e63d2cef4b2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:16.550792 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:16.553793 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.554081 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.554363 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] PATH_INFO: `/v3/users/7e6cae1e5a0f4c23951280814b9f6a13` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.555976 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6cae1e5a0f4c23951280814b9f6a13)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:16.558220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.584826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:16.643021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6cae1e5a0f4c23951280814b9f6a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:16.643834 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6cae1e5a0f4c23951280814b9f6a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:16.651349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6cae1e5a0f4c23951280814b9f6a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:16.663744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6cae1e5a0f4c23951280814b9f6a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:16.678096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6cae1e5a0f4c23951280814b9f6a13'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:16.685865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1381/2731] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:16 2022] DELETE /identity/v3/users/7e6cae1e5a0f4c23951280814b9f6a13 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:16.690345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec563e-3538-4557-859c-e0848043241d None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.690838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec563e-3538-4557-859c-e0848043241d None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.691298 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec563e-3538-4557-859c-e0848043241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.711408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-60ec563e-3538-4557-859c-e0848043241d None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:16.712990 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-60ec563e-3538-4557-859c-e0848043241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:16.792093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1351/2732] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:55:16.802830 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:16.804327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:16.895899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc872-bf68-4647-9a2b-3cb18f307d62 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:16.900490 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:16.901157 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:16.901442 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:16.901720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:16.919539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:16.920317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:16.934084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1382/2733] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:17.367984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec563e-3538-4557-859c-e0848043241d None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:17.370571 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-60ec563e-3538-4557-859c-e0848043241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:17.392531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec563e-3538-4557-859c-e0848043241d None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:17.402841 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:17.403912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:17.404263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:17.406691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] PATH_INFO: `/v3/projects/e3658c9591db45b48802ddebdd1deb39` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:17.410566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3658c9591db45b48802ddebdd1deb39)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:17.417471 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:17.494608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:17.505109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3658c9591db45b48802ddebdd1deb39', 'request_id': 'req-817d357c-21fa-4ff5-9167-e976faae89d5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.512972 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3658c9591db45b48802ddebdd1deb39', 'request_id': 'req-817d357c-21fa-4ff5-9167-e976faae89d5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.524982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3658c9591db45b48802ddebdd1deb39', 'request_id': 'req-817d357c-21fa-4ff5-9167-e976faae89d5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.541511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] The token cache is being invalidate because project e3658c9591db45b48802ddebdd1deb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:17.541511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.541511 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.543334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1352/2734] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:17 2022] DELETE /identity/v3/projects/e3658c9591db45b48802ddebdd1deb39 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:17.546587 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:17.559689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bd1f0-dca0-44cf-92f4-8faaa8c42494 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:17.637086 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:17.638336 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:17.638622 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:17.638903 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] PATH_INFO: `/v3/users/29303317c8454fdf9f0e18824534e6a1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:17.640311 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29303317c8454fdf9f0e18824534e6a1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:17.646128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:17.712021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:17.730727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29303317c8454fdf9f0e18824534e6a1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.732196 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29303317c8454fdf9f0e18824534e6a1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.746668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29303317c8454fdf9f0e18824534e6a1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.772540 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29303317c8454fdf9f0e18824534e6a1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.786482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a8073996-5b23-40c0-9ebf-802eaee988e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29303317c8454fdf9f0e18824534e6a1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:17.794993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1383/2735] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:17 2022] DELETE /identity/v3/users/29303317c8454fdf9f0e18824534e6a1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:55:18.146930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.148529 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:18.162936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d357c-21fa-4ff5-9167-e976faae89d5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.233197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.234114 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.234403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.234682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] PATH_INFO: `/v3/projects/a5f32cbfc0ef49f99420863c17ede421` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.235924 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f32cbfc0ef49f99420863c17ede421)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.239869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.299950 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f32cbfc0ef49f99420863c17ede421', 'request_id': 'req-07366c94-df3c-4840-b7d6-608ef7763cb0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:18.309788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f32cbfc0ef49f99420863c17ede421', 'request_id': 'req-07366c94-df3c-4840-b7d6-608ef7763cb0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:18.324254 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f32cbfc0ef49f99420863c17ede421', 'request_id': 'req-07366c94-df3c-4840-b7d6-608ef7763cb0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:18.338423 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] The token cache is being invalidate because project a5f32cbfc0ef49f99420863c17ede421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:18.338842 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:18.339758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-07366c94-df3c-4840-b7d6-608ef7763cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:18.342089 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1353/2736] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:18 2022] DELETE /identity/v3/projects/a5f32cbfc0ef49f99420863c17ede421 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:18.346980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.347409 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.347778 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.367772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:18.369183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:18.443448 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1384/2737] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:55:18.451264 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.451796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.451894 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.471791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:18.473106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:18.485567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1354/2738] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:18.486805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.497274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d94d4-7539-41e1-a9e3-18a95d504db0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.501932 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.502724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.503096 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.503434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.504242 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:55:18.504615 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.506323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.524879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1385/2739] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:55:18.526094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.536964 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43c2da-8201-4d26-9cc3-7657b911def6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.542640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.547131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.547131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.547131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.547131 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.547725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.581612 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1355/2740] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:55:18 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:18.585941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.597002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa103b-1605-417b-baa5-b375eb675dd7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.681446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.682121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.682414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.682695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.683438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.685368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.723223 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1386/2741] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:55:18 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:18.728115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.739662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0503472-ba8d-4725-9f10-42a169d066b8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.750474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.751205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.751497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.751780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.752526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.759134 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.771878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1356/2742] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:55:18.775732 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.788003 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f4162-0b71-4297-866e-aabb5cf9d34f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.793656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.794412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.794765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.795150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.796207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.798684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd10b5-6740-4800-8bb3-63e59d321cd7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.823757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1387/2743] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:55:18.825368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:18.838162 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8eb0e-d979-4a6e-8800-d8a7f8b98924 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:18.843358 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:18.844285 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.844584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.844850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] PATH_INFO: `/v3/projects/75db6694dff5491badc91b2b944e6871/users/9bd4694621b34c678f01f5bc6ba7ce79/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.866097 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75db6694dff5491badc91b2b944e6871, user_id=9bd4694621b34c678f01f5bc6ba7ce79, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:18.868955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:18.874431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:18.886609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1357/2744] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:18 2022] PUT /identity/v3/projects/75db6694dff5491badc91b2b944e6871/users/9bd4694621b34c678f01f5bc6ba7ce79/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:18.897593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:18.898150 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:18.898736 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:18.929201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] MFA Rules not processed for user `9bd4694621b34c678f01f5bc6ba7ce79`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:18.930658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:18.935152 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:19.020729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1388/2745] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:19.025357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.116077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe1428-0107-4ff9-b7d2-93a24974c68d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.121876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.122563 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.122850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.123195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.123912 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.125897 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.162450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1358/2746] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:55:19 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:55:19.166399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.180446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ffb87-c328-4a73-8aaf-e41b2d7293aa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.250684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.251404 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.251695 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.251977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.252685 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.254939 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.279115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1389/2747] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:55:19 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:55:19.282735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.293929 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc58d51-897d-4e9f-90c2-eae8bb826e6f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.298858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.301295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.301295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.301295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.301295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.302686 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.314825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1359/2748] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:19.321444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.342723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad8f86-6ba1-428b-be5f-0c4ce6ca347f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.351035 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.352828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.353244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.354023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] PATH_INFO: `/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2/users/b6c30da049e4406eb0d46a37dfdd9271/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.396765 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea53b6c155e64ca5a8e8b4c15e6d46d2, user_id=b6c30da049e4406eb0d46a37dfdd9271, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.401121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.421977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1390/2749] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:19 2022] PUT /identity/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2/users/b6c30da049e4406eb0d46a37dfdd9271/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:19.426112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.437256 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7f4f-1c18-472c-abf5-f65b7dd141cd admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.497119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.497799 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.498093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.498374 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.499255 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.504796 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793cc2-4bf3-46d5-b8f6-80ac3bd46f1f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.519094 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1360/2750] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:55:19.520752 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.532350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26f8b7-926f-4ec7-b0ec-35d6fa096002 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.539897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.541160 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.541542 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.541892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] PATH_INFO: `/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2/users/b6c30da049e4406eb0d46a37dfdd9271/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.553226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea53b6c155e64ca5a8e8b4c15e6d46d2, user_id=b6c30da049e4406eb0d46a37dfdd9271, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.553226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.564009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1391/2751] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:55:19 2022] PUT /identity/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2/users/b6c30da049e4406eb0d46a37dfdd9271/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:19.576771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178f3da-0638-4e7e-8873-14d793b84557 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.577324 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178f3da-0638-4e7e-8873-14d793b84557 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.577679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178f3da-0638-4e7e-8873-14d793b84557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.605748 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9178f3da-0638-4e7e-8873-14d793b84557 None None] MFA Rules not processed for user `b6c30da049e4406eb0d46a37dfdd9271`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:19.606969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9178f3da-0638-4e7e-8873-14d793b84557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:19.661056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:19.681763 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1361/2752] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:55:19.693204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.695124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:19.747205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.747522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.747825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.753913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615d8c-c213-4b2e-b44d-98ecfd684db6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.761263 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.762049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.763247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.763247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.769142 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] MFA Rules not processed for user `690436e83b26412cb5c84f47202834f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:19.770389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:19.773337 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.774842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.780137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:19.785527 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1392/2753] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:19.830164 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1362/2754] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:19.866681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:19.875718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ca2e-c4a4-4992-b07e-acc287e9d0dd service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:19.879825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:19.880465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:19.880748 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:19.881024 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:19.940746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:19.942261 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89486e10-5bd5-4d6c-b459-207ac15ee57c service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:19.952077 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1393/2755] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:20.123243 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:20.125007 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:20.197087 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b20214-4533-4ea2-90a7-5776865ed907 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:20.201840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:20.202636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:20.203083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:20.203406 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:20.206010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:20.216635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:20.218080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:20.227903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1363/2756] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:22.006296 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:22.006825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:22.006825 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:22.025306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:22.026524 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:22.085537 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1394/2757] 10.222.0.74 () {58 vars in 1009 bytes} [Thu Jun 16 16:55:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:22.098536 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:22.100000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:22.185642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412405a-a4ad-4388-b7c1-378869b5e3b5 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:22.190988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:22.191674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:22.191958 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:22.192230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:22.201001 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:22.202599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c0b56-9cfe-4bc1-bff2-65d35b26b879 service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:22.210709 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1364/2758] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:55:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:55:24.943412 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:25.010329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b8748-8df5-4148-99d0-8972caba20ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:25.019357 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:25.020267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:25.020547 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:25.020819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] PATH_INFO: `/v3/users/9bd4694621b34c678f01f5bc6ba7ce79` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:25.022191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd4694621b34c678f01f5bc6ba7ce79)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:25.024723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:25.087538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:25.088980 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd4694621b34c678f01f5bc6ba7ce79'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.090184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd4694621b34c678f01f5bc6ba7ce79'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.101221 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd4694621b34c678f01f5bc6ba7ce79'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.114799 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd4694621b34c678f01f5bc6ba7ce79'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.125184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd4694621b34c678f01f5bc6ba7ce79'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.134750 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1395/2759] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:24 2022] DELETE /identity/v3/users/9bd4694621b34c678f01f5bc6ba7ce79 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:25.139662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:25.140155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:25.140443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:25.158318 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:55:25.159568 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:25.168918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:25.234547 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1365/2760] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:55:25.241278 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:25.244053 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:25.314151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa094-ca3c-48e7-9a94-6250fbc65c56 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:25.318021 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:25.318755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:25.319137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:25.319482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:25.329707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:25.331312 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:25.343620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1396/2761] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:25.695903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:25.697338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:25.714160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e3c8-dfc7-4009-97ea-012e116ed369 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:25.721205 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:25.722228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:25.722579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:25.722999 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] PATH_INFO: `/v3/projects/75db6694dff5491badc91b2b944e6871` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:25.724567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75db6694dff5491badc91b2b944e6871)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:25.727915 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:25.795738 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75db6694dff5491badc91b2b944e6871', 'request_id': 'req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.801402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75db6694dff5491badc91b2b944e6871', 'request_id': 'req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.811720 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75db6694dff5491badc91b2b944e6871', 'request_id': 'req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.825505 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] The token cache is being invalidate because project 75db6694dff5491badc91b2b944e6871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:25.826005 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.827231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.832222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1366/2762] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:25 2022] DELETE /identity/v3/projects/75db6694dff5491badc91b2b944e6871 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:25.833459 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:25.844659 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f86f02-54d5-49bd-acad-3bf2adca67ae service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:25.923000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:25.923925 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:25.924206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:25.924480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] PATH_INFO: `/v3/users/b6c30da049e4406eb0d46a37dfdd9271` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:25.925940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c30da049e4406eb0d46a37dfdd9271)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:25.928156 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:25.985199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c30da049e4406eb0d46a37dfdd9271'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.986346 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c30da049e4406eb0d46a37dfdd9271'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:25.997936 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c30da049e4406eb0d46a37dfdd9271'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.006719 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:26.013151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c30da049e4406eb0d46a37dfdd9271'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.022831 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c30da049e4406eb0d46a37dfdd9271'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.032560 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1397/2763] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:55:25 2022] DELETE /identity/v3/users/b6c30da049e4406eb0d46a37dfdd9271 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:26.317195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:26.318773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:26.340359 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e2f7-5a0b-475c-9dbc-532a367e16e3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:26.413309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:26.414276 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:26.414559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:26.414840 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] PATH_INFO: `/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:26.416102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea53b6c155e64ca5a8e8b4c15e6d46d2)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:26.418043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:26.459643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:26.470619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea53b6c155e64ca5a8e8b4c15e6d46d2', 'request_id': 'req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.476447 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea53b6c155e64ca5a8e8b4c15e6d46d2', 'request_id': 'req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.489488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea53b6c155e64ca5a8e8b4c15e6d46d2', 'request_id': 'req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.504102 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] The token cache is being invalidate because project ea53b6c155e64ca5a8e8b4c15e6d46d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:55:26.504379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.505115 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:55:26.506759 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1367/2764] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:55:26 2022] DELETE /identity/v3/projects/ea53b6c155e64ca5a8e8b4c15e6d46d2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:55:33.774067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:33.775729 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:33.838985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52901269-ea2e-4a65-840b-deb8f27070ae admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:33.892071 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:33.892744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:33.893032 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:33.893310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:33.895397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:33.988149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:33.989602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:34.010637 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1398/2765] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:55:36.500856 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:44.008240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:55:55.021067 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:55:55.022810 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:55.036682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51857bd0-b670-4dfc-89f7-3c5a97a66cfc admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:55:55.040807 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:55:55.041457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:55:55.041743 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:55:55.042029 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:55:55.043983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:55:55.113427 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:55:55.115798 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:55:55.127088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1368/2766] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:55:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:05.122551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=30 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:12.972599 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:12.982153 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481f6d-1299-47c3-945a-ec4959b840cf service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:12.986506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:12.987193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:12.987513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:12.987794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:12.989698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:13.078458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:13.080020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:13.100359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1399/2767] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:22.086037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:22.095190 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae03e0-1671-4c53-b186-039a47e9e69f service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:22.098765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:22.099528 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:22.100056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:22.100056 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:22.101948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:22.120594 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:22.122288 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:22.131892 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1369/2768] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:23.097026 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=41 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:26.147675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:26.158554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7ef1-e3a5-43da-a7d5-d598c8ce8561 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:26.162300 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:26.162953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:26.163271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:26.163549 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:26.165401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:26.232171 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:26.233649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:26.244271 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1400/2769] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:32.127543 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=6 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:32.203560 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:32.205525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:32.287570 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bed56-b777-48e6-b033-cad5b93d7780 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:32.291262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:32.291928 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:32.292210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:32.292508 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:32.294670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:32.375700 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:32.377222 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:32.387698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1370/2770] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:56:32.982696 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:32.986193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:33.056566 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:33.058261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:33.073410 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154432db-96db-4457-9aa9-f65e4075dacc service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:33.077662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1b9c5-4c75-4268-bff9-cb4e4ce8e633 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:33.078502 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:33.079519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:33.079852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:33.080137 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] PATH_INFO: `/v3/users/d162fcafed2c4ae099a5786cceb84891` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:33.081308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d162fcafed2c4ae099a5786cceb84891)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:33.082772 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:33.083331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:33.083744 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:33.084046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:33.084323 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:33.086129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:33.098122 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:33.099958 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:33.110468 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1401/2771] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:33.136740 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd162fcafed2c4ae099a5786cceb84891'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.137827 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd162fcafed2c4ae099a5786cceb84891'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.146546 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd162fcafed2c4ae099a5786cceb84891'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.153770 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd162fcafed2c4ae099a5786cceb84891'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.165836 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd162fcafed2c4ae099a5786cceb84891'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.172173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1371/2772] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:56:33 2022] DELETE /identity/v3/users/d162fcafed2c4ae099a5786cceb84891 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:56:33.352129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:33.363212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46182983-525a-4fa2-a231-2a47bf9f8f79 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:33.427131 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:33.428181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:33.428485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:33.428758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] PATH_INFO: `/v3/projects/256a9b5f5ad1405bb9fe21d591ef677d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:33.429983 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256a9b5f5ad1405bb9fe21d591ef677d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:33.431968 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:33.463709 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:33.479862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256a9b5f5ad1405bb9fe21d591ef677d', 'request_id': 'req-9e251393-bffc-4523-84c8-da04e10d5935'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.487879 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256a9b5f5ad1405bb9fe21d591ef677d', 'request_id': 'req-9e251393-bffc-4523-84c8-da04e10d5935'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.494517 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256a9b5f5ad1405bb9fe21d591ef677d', 'request_id': 'req-9e251393-bffc-4523-84c8-da04e10d5935'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.505441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] The token cache is being invalidate because project 256a9b5f5ad1405bb9fe21d591ef677d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:56:33.505901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.506618 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.508450 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1402/2773] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:56:33 2022] DELETE /identity/v3/projects/256a9b5f5ad1405bb9fe21d591ef677d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:56:33.512940 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:33.514477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:33.528011 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524797c-02c9-49a5-9919-3d3fb12d7161 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:33.553086 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:33.587074 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:33.587956 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:33.588234 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:33.588504 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] PATH_INFO: `/v3/users/be1f4af420984092a27be3409b162ab4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:33.601982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1f4af420984092a27be3409b162ab4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:33.604014 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:33.649735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1f4af420984092a27be3409b162ab4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.650372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1f4af420984092a27be3409b162ab4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.657877 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1f4af420984092a27be3409b162ab4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.665350 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1f4af420984092a27be3409b162ab4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.679736 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-d44cf4ca-4c27-4b54-826e-b41d4b4b1555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1f4af420984092a27be3409b162ab4'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:33.686825 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1372/2774] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:56:33 2022] DELETE /identity/v3/users/be1f4af420984092a27be3409b162ab4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:56:33.880354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:33.881791 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:33.895277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e251393-bffc-4523-84c8-da04e10d5935 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:33.956657 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:33.957543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:33.957829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:33.958286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] PATH_INFO: `/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:33.970401 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dc3dea81894b6ab41ae93f1c48693c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:33.972381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:34.019145 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dc3dea81894b6ab41ae93f1c48693c', 'request_id': 'req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:34.026872 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dc3dea81894b6ab41ae93f1c48693c', 'request_id': 'req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:34.033955 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dc3dea81894b6ab41ae93f1c48693c', 'request_id': 'req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:34.044574 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] The token cache is being invalidate because project d6dc3dea81894b6ab41ae93f1c48693c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:56:34.044865 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:34.045624 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:56:34.047210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1403/2775] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:56:33 2022] DELETE /identity/v3/projects/d6dc3dea81894b6ab41ae93f1c48693c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:56:38.309275 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:38.309828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:38.309828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:38.328341 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] MFA Rules not processed for user `d5d5414645ec47e1bcaa56860ba6daf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:56:38.329556 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:38.388488 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1373/2776] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:56:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:56:39.014431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:39.015990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:39.061116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:39.095175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536d9f-5bf2-42e4-8e1a-3d2e1be110dd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:39.101649 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:39.102331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:39.102610 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:39.102910 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:39.105115 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:39.119000 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:39.120694 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:39.132620 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1404/2777] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:48.303385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:56:48.304979 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:48.356124 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:48.404733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225bf84-7816-4b03-a926-30f9f3df3bb9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:56:48.410616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:56:48.411590 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:56:48.411878 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:56:48.412150 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:56:48.414139 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:56:48.432562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:56:48.434165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:56:48.446263 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1374/2778] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:56:49.127189 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=16 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:56:58.441299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=16 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:29.748621 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:29.762561 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a07126-4295-42de-86cd-b5a98d114d4c service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:29.768566 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:29.769214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:29.769498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:29.769774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:29.771751 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:29.784394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:29.785977 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec14192-4a75-46bf-b69b-365514d95490 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:29.797661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1405/2779] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:35.095843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:35.111486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666b905-8363-4d9e-9a64-5f33450a915e service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:35.117723 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6297dc028a44750a69132257e0aede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:35.118386 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:35.118698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:35.118959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:35.120808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:35.142988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:35.144614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:35.156452 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1375/2780] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:58:36.192829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:36.193394 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:36.194039 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:36.214878 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] MFA Rules not processed for user `687bbc1e0d174b78a51d01542d1fd03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:36.216142 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:36.281104 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1406/2781] 10.222.0.74 () {58 vars in 967 bytes} [Thu Jun 16 16:58:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:36.322611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:36.324207 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:36.398184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b538a-ad79-4470-a612-6b6d6b297338 service nova] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:36.404535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fc2a725e2f14b90be5575a030f47dae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:36.405201 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:36.405489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:36.405767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:36.414627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:36.416283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:36.424525 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1376/2782] 10.222.0.74 () {60 vars in 1384 bytes} [Thu Jun 16 16:58:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:58:43.720202 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:43.724178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:43.808067 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63238-818e-4624-96a0-25a108c0af0e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:43.815688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:43.816580 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:43.816847 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:43.817136 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] PATH_INFO: `/v3/users/690436e83b26412cb5c84f47202834f4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:43.818664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=690436e83b26412cb5c84f47202834f4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:43.820707 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:43.879444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690436e83b26412cb5c84f47202834f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:43.880128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690436e83b26412cb5c84f47202834f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:43.888127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690436e83b26412cb5c84f47202834f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:43.900548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690436e83b26412cb5c84f47202834f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:43.908974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690436e83b26412cb5c84f47202834f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:43.915190 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1407/2783] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:43 2022] DELETE /identity/v3/users/690436e83b26412cb5c84f47202834f4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:44.110112 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:44.121377 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214961a-0302-4156-8f32-13fcab54c4a5 service swift] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:44.183657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:44.184551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:44.184898 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:44.185309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] PATH_INFO: `/v3/projects/0a55a4c28c4140958e1c078eaf57f9fd` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:44.186633 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a55a4c28c4140958e1c078eaf57f9fd)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:44.188599 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:44.237390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a55a4c28c4140958e1c078eaf57f9fd', 'request_id': 'req-6d17490e-2801-41ed-9022-db786b5818ee'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.244345 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a55a4c28c4140958e1c078eaf57f9fd', 'request_id': 'req-6d17490e-2801-41ed-9022-db786b5818ee'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.254553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a55a4c28c4140958e1c078eaf57f9fd', 'request_id': 'req-6d17490e-2801-41ed-9022-db786b5818ee'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.267875 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] The token cache is being invalidate because project 0a55a4c28c4140958e1c078eaf57f9fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:58:44.268196 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.268839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.270415 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1377/2784] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:58:44 2022] DELETE /identity/v3/projects/0a55a4c28c4140958e1c078eaf57f9fd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:44.274795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:44.276705 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:44.290185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cca8fe-14d7-4d84-a796-bc811ba73706 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:44.298503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:44.353721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:44.355354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:44.355354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:44.355354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] PATH_INFO: `/v3/users/bd0c5158f950442dabe9781948ea8fb2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:44.356645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0c5158f950442dabe9781948ea8fb2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:44.358734 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:44.411288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0c5158f950442dabe9781948ea8fb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.411954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0c5158f950442dabe9781948ea8fb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.419565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0c5158f950442dabe9781948ea8fb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.432244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0c5158f950442dabe9781948ea8fb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.440306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0c5158f950442dabe9781948ea8fb2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.448954 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1408/2785] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:44 2022] DELETE /identity/v3/users/bd0c5158f950442dabe9781948ea8fb2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:44.776983 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:44.778562 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:44.791741 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17490e-2801-41ed-9022-db786b5818ee admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:44.795643 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:44.863826 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:44.864871 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:44.865174 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:44.865465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] PATH_INFO: `/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:44.866801 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d522f3b82f84bd78aeeec19b2e49e5e)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:44.868778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:44.926575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d522f3b82f84bd78aeeec19b2e49e5e', 'request_id': 'req-74600f62-2fc6-44b9-ab71-e7d9a11361a0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.932702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d522f3b82f84bd78aeeec19b2e49e5e', 'request_id': 'req-74600f62-2fc6-44b9-ab71-e7d9a11361a0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.941221 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d522f3b82f84bd78aeeec19b2e49e5e', 'request_id': 'req-74600f62-2fc6-44b9-ab71-e7d9a11361a0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.953851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] The token cache is being invalidate because project 1d522f3b82f84bd78aeeec19b2e49e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:58:44.954132 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.954761 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:44.956316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1378/2786] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:58:44 2022] DELETE /identity/v3/projects/1d522f3b82f84bd78aeeec19b2e49e5e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:58:49.954998 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:49.956465 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:50.018413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935f24e-c489-430f-a1aa-72d333c38c48 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:50.068389 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:50.076120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:50.076797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:50.077643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:50.077937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:50.080543 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:50.148945 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:50.150307 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:50.160933 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1409/2787] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:50.289236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:50.290732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:50.364690 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74600f62-2fc6-44b9-ab71-e7d9a11361a0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:50.371500 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:50.372188 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:50.372477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:50.372755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:50.374811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:50.389004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:50.390641 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:50.403267 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1379/2788] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:53.416433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:53.417920 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:53.431673 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7ac51-b099-4c68-9730-6889233b6a9c service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:53.436676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:53.437826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:53.437960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:53.438138 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] PATH_INFO: `/v3/users/d5d5414645ec47e1bcaa56860ba6daf6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:53.439710 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d5414645ec47e1bcaa56860ba6daf6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:53.442121 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:53.496767 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d5414645ec47e1bcaa56860ba6daf6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.497472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d5414645ec47e1bcaa56860ba6daf6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.506288 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d5414645ec47e1bcaa56860ba6daf6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.520584 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d5414645ec47e1bcaa56860ba6daf6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.532228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-84338beb-e022-40fc-89ae-feec7647b146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d5414645ec47e1bcaa56860ba6daf6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.541565 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1410/2789] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:53 2022] DELETE /identity/v3/users/d5d5414645ec47e1bcaa56860ba6daf6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:53.748805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:53.760988 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3797ea-cd5e-47f0-80e1-3d52f2bcab61 service placement] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:53.794338 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:53.843891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:53.844782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:53.845083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:53.845366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] PATH_INFO: `/v3/projects/992e429cb62643a1885c29538604ae5d` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:53.846587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=992e429cb62643a1885c29538604ae5d)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:53.848889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:53.909781 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992e429cb62643a1885c29538604ae5d', 'request_id': 'req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.917293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992e429cb62643a1885c29538604ae5d', 'request_id': 'req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.926465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992e429cb62643a1885c29538604ae5d', 'request_id': 'req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.938479 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] The token cache is being invalidate because project 992e429cb62643a1885c29538604ae5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:58:53.938771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.939537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-4aa3436a-85e4-46fe-9f0a-f429fcb373f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:53.941284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1380/2790] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:58:53 2022] DELETE /identity/v3/projects/992e429cb62643a1885c29538604ae5d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:53.946717 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:53.947011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:53.947333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:53.966376 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:53.967644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:54.031712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1411/2791] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:54.040473 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.040772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.041061 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.061189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:54.062430 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:54.070984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1381/2792] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:54.075451 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.086698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119540cd-5ca5-4b44-a05a-0abbbf640c7f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.092718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.093413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.093698 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.093974 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.094718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:58:54.095030 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.096803 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.099681 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:54.110801 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1412/2793] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:58:54.115392 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.125357 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c42111-0fd5-4b74-8ac8-af8585999db6 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.130312 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.131366 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.131750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.132259 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.132982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.134914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.180767 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1382/2794] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:58:54 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:54.187503 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.199569 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ce23f-58a7-45fe-a461-3bf2389e49e5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.264185 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.264860 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.265154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.265434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.266123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.268105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.290597 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1413/2795] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:58:54 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:54.294998 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.306772 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cba00-98b2-48bb-889a-82764834aab5 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.311698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.312376 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.312665 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.312970 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.313710 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.315418 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.325921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1383/2796] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:54.330194 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.340995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0ce2-d727-426d-b0b3-da9cdc321997 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.345889 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.346554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.346839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.347182 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.347919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.349570 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbdf4-649d-4428-919a-009f9e66fe9c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.360446 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1414/2797] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:54.364497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.375588 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879216c-8e15-4427-ba84-612c61b85c8c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.381691 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.382597 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.382881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.383195 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] PATH_INFO: `/v3/projects/e90b02afb7a14eec990ee08db82e027d/users/a8f2f38152684cf1848c951d7f280243/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.399129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e90b02afb7a14eec990ee08db82e027d, user_id=a8f2f38152684cf1848c951d7f280243, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.401782 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.419755 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1384/2798] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:54 2022] PUT /identity/v3/projects/e90b02afb7a14eec990ee08db82e027d/users/a8f2f38152684cf1848c951d7f280243/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:54.427126 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525400ad-8ae5-4767-9bb3-c7279435ab95 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.428244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525400ad-8ae5-4767-9bb3-c7279435ab95 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.428244 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525400ad-8ae5-4767-9bb3-c7279435ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.452990 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-525400ad-8ae5-4767-9bb3-c7279435ab95 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:54.454228 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-525400ad-8ae5-4767-9bb3-c7279435ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:54.518907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1415/2799] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:54.529080 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:54.530939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:54.585766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:54.599389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e68c8-9653-4003-bd8f-d28f31ae9868 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:54.603592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:54.604252 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:54.604535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:54.604811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:54.617226 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:54.618679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:54.628808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1385/2800] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:55.269843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.270151 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.270424 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.298551 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] MFA Rules not processed for user `a8f2f38152684cf1848c951d7f280243`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:55.299804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:55.376629 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1416/2801] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:55 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:55.380173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.391090 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dced61-5875-45e7-a96b-23ee9b4c8dd5 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.395895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.397668 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.397919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.398334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.399140 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.401129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.432363 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1386/2802] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:58:55 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:55.434051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.442199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:55.448220 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ef874-2f6f-45e3-8667-069be5878b61 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.513123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.513836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.514116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.514386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.515141 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.517116 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.539178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1417/2803] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:58:55 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:55.543211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.556428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da174ce0-10da-451c-8052-ff17a581c62b admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.561833 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.562519 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.562805 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.563111 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.563853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.565472 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.576971 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1387/2804] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:55.581248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.592458 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bca8a6-87b8-443d-bac9-0e20a4e574a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.597658 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.598554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.598838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.599107 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] PATH_INFO: `/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593/users/8246d5b3c74246868daf63c282ba1636/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.615672 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b38a9b51da45fe9c1d1c19a6a56593, user_id=8246d5b3c74246868daf63c282ba1636, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.618428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.633227 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1418/2805] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:55 2022] PUT /identity/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593/users/8246d5b3c74246868daf63c282ba1636/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:55.637683 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.649127 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03ce25-3c7c-4e1e-abc7-bc5fe5d576a9 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.711394 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.712099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.712385 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.712650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.713421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.715035 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bda23-8a13-4599-8d8c-b1ef775eedb6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.731620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1388/2806] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:55.736117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:55.747643 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0541b-1b8b-45fc-b6c2-5c9136be5889 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:55.752745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:55.753641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:55.753926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:55.754205 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] PATH_INFO: `/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593/users/8246d5b3c74246868daf63c282ba1636/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:55.756501 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b38a9b51da45fe9c1d1c19a6a56593, user_id=8246d5b3c74246868daf63c282ba1636, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:55.759276 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:55.770640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1419/2807] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:55 2022] PUT /identity/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593/users/8246d5b3c74246868daf63c282ba1636/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:56.406829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:56.407432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:56.407432 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:56.438638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] MFA Rules not processed for user `8246d5b3c74246868daf63c282ba1636`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:56.440057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:56.528402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1389/2808] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:56.994947 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:56.996919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:57.083421 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d23da-e843-4999-b16a-e0e7daf448d4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:57.091592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:57.092169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:57.092457 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:57.092738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:57.094866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:57.107995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:57.110051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:57.120913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1420/2809] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:57.595982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:57.640118 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:57.663681 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d914-965b-4421-9d66-8104a6622cdd None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:57.668246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:57.669129 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:57.669409 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:57.669682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] PATH_INFO: `/v3/users/a8f2f38152684cf1848c951d7f280243` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:57.670862 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f2f38152684cf1848c951d7f280243)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:57.672829 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:57.718072 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f2f38152684cf1848c951d7f280243'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:57.718694 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f2f38152684cf1848c951d7f280243'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:57.726210 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f2f38152684cf1848c951d7f280243'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:57.733069 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f2f38152684cf1848c951d7f280243'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:57.745974 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f2f38152684cf1848c951d7f280243'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:57.752334 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1390/2810] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:57 2022] DELETE /identity/v3/users/a8f2f38152684cf1848c951d7f280243 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:57.933741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:57.945280 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301fb47-7ed7-486d-be82-2fb884ce2fcb service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:57.991940 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:58.007460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.008274 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.008557 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.008839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] PATH_INFO: `/v3/projects/e90b02afb7a14eec990ee08db82e027d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.010386 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e90b02afb7a14eec990ee08db82e027d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.012368 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.072661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e90b02afb7a14eec990ee08db82e027d', 'request_id': 'req-6c5ade45-5e92-4c93-9686-6f6269aca5da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.081727 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e90b02afb7a14eec990ee08db82e027d', 'request_id': 'req-6c5ade45-5e92-4c93-9686-6f6269aca5da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.089283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e90b02afb7a14eec990ee08db82e027d', 'request_id': 'req-6c5ade45-5e92-4c93-9686-6f6269aca5da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.102137 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] The token cache is being invalidate because project e90b02afb7a14eec990ee08db82e027d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:58:58.102427 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.103180 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.104797 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1421/2811] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:58:57 2022] DELETE /identity/v3/projects/e90b02afb7a14eec990ee08db82e027d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:58.108858 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.110309 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:58.123407 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9884-a410-407c-871c-eb2523032b82 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.185552 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.186431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.186708 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.186984 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] PATH_INFO: `/v3/users/8246d5b3c74246868daf63c282ba1636` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.188236 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8246d5b3c74246868daf63c282ba1636)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.190155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.238542 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8246d5b3c74246868daf63c282ba1636'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.239212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8246d5b3c74246868daf63c282ba1636'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.246784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8246d5b3c74246868daf63c282ba1636'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.248942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:58.254689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8246d5b3c74246868daf63c282ba1636'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.268419 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-8a4b614b-e4b9-42d0-a02c-77ea8f4fed0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8246d5b3c74246868daf63c282ba1636'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.274464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1391/2812] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:58 2022] DELETE /identity/v3/users/8246d5b3c74246868daf63c282ba1636 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:58.455839 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.457438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:58.474293 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ade45-5e92-4c93-9686-6f6269aca5da admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.535072 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.535884 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.536165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.536438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] PATH_INFO: `/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.537836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b38a9b51da45fe9c1d1c19a6a56593)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.539794 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.587623 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b38a9b51da45fe9c1d1c19a6a56593', 'request_id': 'req-476d1d43-8f96-4570-848e-aee0557f3c25'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.596665 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b38a9b51da45fe9c1d1c19a6a56593', 'request_id': 'req-476d1d43-8f96-4570-848e-aee0557f3c25'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.603573 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b38a9b51da45fe9c1d1c19a6a56593', 'request_id': 'req-476d1d43-8f96-4570-848e-aee0557f3c25'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.614942 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] The token cache is being invalidate because project 92b38a9b51da45fe9c1d1c19a6a56593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:58:58.615414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.616266 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-476d1d43-8f96-4570-848e-aee0557f3c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:58:58.617898 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1422/2813] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:58:58 2022] DELETE /identity/v3/projects/92b38a9b51da45fe9c1d1c19a6a56593 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:58.624009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.624526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.624526 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.644534 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:58.645774 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:58.711778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1392/2814] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:58.719787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.720070 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.720349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.733283 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:58.740166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:58.742630 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:58.752023 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1423/2815] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:58.757347 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.768143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ced9e-aaac-431f-80c0-a1dcac8262e7 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.774096 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.774818 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.775130 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.775414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.776161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:58:58.776474 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.778119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.792420 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1393/2816] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:58:58.796995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.807163 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf104-ea90-4669-b2d0-1ba62b92e639 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.811975 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.812668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.812953 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.813230 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.813926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.815887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.845528 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1424/2817] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:58:58 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:58.851354 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.861559 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4776f-d953-4f1c-a8b4-5ee794f4c29d admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.925789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.926486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.926747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.927022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.927785 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.929702 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.951206 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1394/2818] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:58:58 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:58:58.955358 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:58.965960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa88d6-333a-4a6e-8bad-78a5282c87f2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:58.971601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:58.972319 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:58.972646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:58.972922 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:58.973661 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:58.976055 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:58.987433 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1425/2819] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:58.995621 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.006466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d5c77-a611-4c53-9fa0-76acb0b7f622 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.011626 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.012292 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.012575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.012868 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.013596 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.015216 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13703550-f3a0-4df4-a08c-caf131b4c87c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.025765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1395/2820] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:59.030148 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.040496 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02287f-d35a-4839-bdf2-5aa4d7d3f154 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.045498 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.046320 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.046602 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.046880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] PATH_INFO: `/v3/projects/ef565a12292842a78c1ac5939b5edfc2/users/dc5d3da3393844f596e0debaa0038d53/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.066513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef565a12292842a78c1ac5939b5edfc2, user_id=dc5d3da3393844f596e0debaa0038d53, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.070017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.087329 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1426/2821] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:59 2022] PUT /identity/v3/projects/ef565a12292842a78c1ac5939b5edfc2/users/dc5d3da3393844f596e0debaa0038d53/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:59.098009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.098295 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.098569 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.127936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] MFA Rules not processed for user `dc5d3da3393844f596e0debaa0038d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:59.129239 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:59.159773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:59.204783 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1396/2822] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:59.210331 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.276475 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a75cf-4766-4140-8e7f-a99014629820 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.281676 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.282354 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.282641 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.282918 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.283646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.285572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.314210 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1427/2823] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:58:59 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:59.318703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.330843 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5390f7-a12e-4513-a3f4-8f8f961dc5e9 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.396375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.397661 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.397945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.398223 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.398978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.401170 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.424326 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1397/2824] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:58:59 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:58:59.428441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.441413 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb5337-cd9a-415b-8cca-3ac7794235fe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.449204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.450117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.450538 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.450960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.451871 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.454008 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.468367 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1428/2825] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:58:59.473589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.485495 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27076269-6f9f-491d-9001-3df445c0cdc6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.490443 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.491603 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.491918 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.492242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] PATH_INFO: `/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475/users/4787f99373904d39adcc127bc03604d3/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.512965 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba952b2dc8c4eaca69d0f83cc82e475, user_id=4787f99373904d39adcc127bc03604d3, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.517032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.533105 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1398/2826] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:59 2022] PUT /identity/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475/users/4787f99373904d39adcc127bc03604d3/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:58:59.537192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.548233 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141e49-2f95-4d98-8fa5-42cd572b2507 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.618445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.619431 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.619730 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.620009 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.620931 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.622851 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8569def-9abd-46c4-aced-1838cdac7c6a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.635862 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1429/2827] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:58:59.640173 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.652697 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022de88-3cb0-4730-bb7f-7f328c9ea1d4 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.658426 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.659356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.659699 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.660032 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] PATH_INFO: `/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475/users/4787f99373904d39adcc127bc03604d3/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.662154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba952b2dc8c4eaca69d0f83cc82e475, user_id=4787f99373904d39adcc127bc03604d3, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.665027 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.676126 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1399/2828] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:58:59 2022] PUT /identity/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475/users/4787f99373904d39adcc127bc03604d3/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:58:59.686628 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.686959 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.687310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.715721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:58:59.719441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] MFA Rules not processed for user `4787f99373904d39adcc127bc03604d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:58:59.721359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:59.805832 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1430/2829] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:58:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:58:59.817497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:58:59.820041 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:58:59.886544 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135f47f-bf12-4a3b-800b-94947d245a05 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:58:59.892273 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:58:59.892942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:58:59.893231 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:58:59.893514 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:58:59.904202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:58:59.905648 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c040e6-1e73-4d79-9776-966738db832e service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:58:59.915442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1400/2830] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:58:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:58:59.988289 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:00.052800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255a375-29cc-4413-ac80-551b38c29391 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:00.057813 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:00.058684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:00.058965 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:00.059267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] PATH_INFO: `/v3/users/dc5d3da3393844f596e0debaa0038d53` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:00.060471 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5d3da3393844f596e0debaa0038d53)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:00.062382 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:00.111117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5d3da3393844f596e0debaa0038d53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.111688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5d3da3393844f596e0debaa0038d53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.119031 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5d3da3393844f596e0debaa0038d53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.132149 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5d3da3393844f596e0debaa0038d53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.141631 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5d3da3393844f596e0debaa0038d53'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.148763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1431/2831] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:58:59 2022] DELETE /identity/v3/users/dc5d3da3393844f596e0debaa0038d53 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:00.153522 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:00.153817 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:00.154120 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:00.173369 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:00.174857 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:00.195784 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:00.249099 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1401/2832] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:00.259183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:00.260917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:00.281684 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:00.324611 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3716e-75f0-4cb6-92ab-4104fa648b02 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:00.328592 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:00.329247 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:00.329534 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:00.329812 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:00.339645 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:00.340868 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:00.351082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1432/2833] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:00.627486 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:00.628980 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:00.642516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cb0b9-a2fd-4eda-aa6e-2076cf6b2809 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:00.648719 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:00.649611 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:00.650043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:00.650284 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] PATH_INFO: `/v3/projects/ef565a12292842a78c1ac5939b5edfc2` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:00.651650 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef565a12292842a78c1ac5939b5edfc2)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:00.653591 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:00.703614 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef565a12292842a78c1ac5939b5edfc2', 'request_id': 'req-911786df-680a-4bdd-b2ec-696dc1a8b644'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.709861 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef565a12292842a78c1ac5939b5edfc2', 'request_id': 'req-911786df-680a-4bdd-b2ec-696dc1a8b644'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.718955 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef565a12292842a78c1ac5939b5edfc2', 'request_id': 'req-911786df-680a-4bdd-b2ec-696dc1a8b644'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.732197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] The token cache is being invalidate because project ef565a12292842a78c1ac5939b5edfc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:00.732482 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.733197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.734809 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1402/2834] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:00 2022] DELETE /identity/v3/projects/ef565a12292842a78c1ac5939b5edfc2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:59:00.738852 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:00.751120 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817a55-cc07-4fae-a39c-b8ef290fd2a8 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:00.813780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:00.814668 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:00.814952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:00.815265 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] PATH_INFO: `/v3/users/4787f99373904d39adcc127bc03604d3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:00.816494 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4787f99373904d39adcc127bc03604d3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:00.818541 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:00.863478 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4787f99373904d39adcc127bc03604d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.864175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4787f99373904d39adcc127bc03604d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.871445 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4787f99373904d39adcc127bc03604d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.884828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4787f99373904d39adcc127bc03604d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.893609 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-48ecb28f-305b-4de2-89b7-3d0fa01731bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4787f99373904d39adcc127bc03604d3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:00.899997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1433/2835] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:00 2022] DELETE /identity/v3/users/4787f99373904d39adcc127bc03604d3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:01.186355 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.187936 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:01.210421 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911786df-680a-4bdd-b2ec-696dc1a8b644 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.275434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.276306 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.276589 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.276837 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] PATH_INFO: `/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.278092 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba952b2dc8c4eaca69d0f83cc82e475)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.280019 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.328465 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba952b2dc8c4eaca69d0f83cc82e475', 'request_id': 'req-127deca3-330a-4e93-a7a9-53529461ea3e'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:01.329670 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:01.336000 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba952b2dc8c4eaca69d0f83cc82e475', 'request_id': 'req-127deca3-330a-4e93-a7a9-53529461ea3e'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:01.344373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba952b2dc8c4eaca69d0f83cc82e475', 'request_id': 'req-127deca3-330a-4e93-a7a9-53529461ea3e'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:01.357584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] The token cache is being invalidate because project 4ba952b2dc8c4eaca69d0f83cc82e475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:01.357882 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:01.358619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-127deca3-330a-4e93-a7a9-53529461ea3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:01.360327 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1403/2836] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:01 2022] DELETE /identity/v3/projects/4ba952b2dc8c4eaca69d0f83cc82e475 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:59:01.366236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.366848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.366848 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.388143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:01.389305 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:01.428519 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:01.456473 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1434/2837] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:01.466136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.466450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.466701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.486211 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:01.487493 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:01.496004 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1404/2838] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:59:01.500491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.511491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be0d7-89d0-4b52-8ac4-d04f3e6d87d8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.517253 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.517917 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.518201 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.518507 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.519292 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:59:01.519601 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.521273 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.534482 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1435/2839] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:59:01.539638 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.550752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437ed55-3d70-4435-81ae-e66b0bcf17f2 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.555995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.556679 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.556948 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.557250 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.558093 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.560100 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.588088 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1405/2840] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:01.592843 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.607598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effbe4d-53fa-46e2-8dcc-a0abeda9ba82 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.668810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.669390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.669671 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.669941 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.670626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.672577 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.693098 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1436/2841] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:01.697724 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.709466 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9bb8f-affc-4e73-8ea9-9ef20dd37d9c admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.714942 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.715765 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.716057 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.716331 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.717064 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.718689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.730283 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1406/2842] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:01.734310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.744441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542eb4-3778-4e47-b8d3-b9d0e786e1b6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.749640 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.750140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.750422 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.750700 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.751441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.753103 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b46f-b678-4625-891c-2e9691467486 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.763651 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1437/2843] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:01.767866 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:01.778676 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76998f8e-7c4b-492a-98b1-457f5b360034 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:01.783389 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:01.784299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.784579 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.784849 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] PATH_INFO: `/v3/projects/ab220316e3a149f9ac935be2e50508c7/users/19c84c714dcc45f99576aa4e9d17691d/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.803637 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab220316e3a149f9ac935be2e50508c7, user_id=19c84c714dcc45f99576aa4e9d17691d, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:01.806268 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:01.820477 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1407/2844] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:01 2022] PUT /identity/v3/projects/ab220316e3a149f9ac935be2e50508c7/users/19c84c714dcc45f99576aa4e9d17691d/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:01.830856 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:01.831186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:01.831548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:01.861712 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] MFA Rules not processed for user `19c84c714dcc45f99576aa4e9d17691d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:01.863165 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:01.946558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1438/2845] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:01.952010 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.025128 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a903ece-0aa1-459c-a3c4-3db070b65ef6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.030251 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.030920 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.031248 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.031530 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.032246 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.034175 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.062687 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1408/2846] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:01 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:59:02.067962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.080349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6420e2-2b85-4abd-aed3-fa88ef0c911c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.118302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:02.146214 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.147193 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.147325 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.147593 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.148290 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.150245 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.170699 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1439/2847] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:02 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:02.174753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.185949 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203eaa6-5d3f-4181-a11d-e91158dbe664 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.191485 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.192161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.192453 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.192778 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.193548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.195262 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.207527 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1409/2848] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:02.212390 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.223078 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1087d-356b-4285-969f-7f660e61bfd7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.229836 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.232692 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.232984 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.233269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] PATH_INFO: `/v3/projects/b8e5b4c1093b4a9394d28bd194c51087/users/91ffefb7e9ab47a8a2847706ec3ebbe8/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.251043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e5b4c1093b4a9394d28bd194c51087, user_id=91ffefb7e9ab47a8a2847706ec3ebbe8, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.253738 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.273598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1440/2849] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:02 2022] PUT /identity/v3/projects/b8e5b4c1093b4a9394d28bd194c51087/users/91ffefb7e9ab47a8a2847706ec3ebbe8/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:02.277704 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.288725 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2040e3-cf5e-4699-a175-9f0ac530d0d6 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.336632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:02.357136 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.358322 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.358851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.359408 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.360651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.363532 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752c944-9c7f-45d8-89d0-f70f1275c353 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.384636 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1410/2850] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:02.389414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.401075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9093cb8-6d67-4ead-ad4d-0e26099193f6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.406741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.407675 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.407926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.408226 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] PATH_INFO: `/v3/projects/b8e5b4c1093b4a9394d28bd194c51087/users/91ffefb7e9ab47a8a2847706ec3ebbe8/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.410449 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e5b4c1093b4a9394d28bd194c51087, user_id=91ffefb7e9ab47a8a2847706ec3ebbe8, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.413762 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.425359 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1441/2851] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:02 2022] PUT /identity/v3/projects/b8e5b4c1093b4a9394d28bd194c51087/users/91ffefb7e9ab47a8a2847706ec3ebbe8/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:02.436619 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.436914 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.437197 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.465773 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] MFA Rules not processed for user `91ffefb7e9ab47a8a2847706ec3ebbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:02.466930 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:02.540659 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1411/2852] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:02.552249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.554117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:02.613110 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb80ba-3724-4e1b-8c54-382cc1518f40 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.617011 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.617655 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.617938 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.618211 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.627687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.629132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.639774 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1442/2853] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:02.745944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:02.761351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a46de-09ba-4da6-a791-9a814e9ff4cc None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:02.765417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:02.766082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:02.766370 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:02.766644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:02.839976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:02.841424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af060d78-7635-4f47-b5e2-081f039e17e4 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:02.850853 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1412/2854] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:05.468178 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:05.540840 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bee39-28fe-40b7-8e7b-3e5f6736f857 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:05.545961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:05.546893 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:05.547194 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:05.547480 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] PATH_INFO: `/v3/users/19c84c714dcc45f99576aa4e9d17691d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:05.548816 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c84c714dcc45f99576aa4e9d17691d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:05.551254 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:05.592043 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:05.607333 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c84c714dcc45f99576aa4e9d17691d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:05.608349 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c84c714dcc45f99576aa4e9d17691d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:05.616598 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c84c714dcc45f99576aa4e9d17691d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:05.629215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c84c714dcc45f99576aa4e9d17691d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:05.638015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c84c714dcc45f99576aa4e9d17691d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:05.651184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1443/2855] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:05 2022] DELETE /identity/v3/users/19c84c714dcc45f99576aa4e9d17691d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:05.663961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:05.663961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:05.663961 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:05.682351 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:05.683808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:05.747300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1413/2856] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:05.758544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:05.760754 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:05.822441 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187505e2-ff78-454d-ac81-3baac86d3893 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:05.827204 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:05.827876 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:05.828159 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:05.828439 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:05.840277 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:05.842014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:05.854105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1444/2857] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:06.138325 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:06.139923 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:06.152945 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32142c34-8d73-4e9b-ae1b-18dea08a404e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:06.158026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:06.158995 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:06.159450 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:06.159789 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] PATH_INFO: `/v3/projects/ab220316e3a149f9ac935be2e50508c7` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:06.161028 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab220316e3a149f9ac935be2e50508c7)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:06.163340 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:06.166553 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=100 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:06.218828 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab220316e3a149f9ac935be2e50508c7', 'request_id': 'req-786db3b8-7577-4c46-94cf-b115e698abcb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.225662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab220316e3a149f9ac935be2e50508c7', 'request_id': 'req-786db3b8-7577-4c46-94cf-b115e698abcb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.234095 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab220316e3a149f9ac935be2e50508c7', 'request_id': 'req-786db3b8-7577-4c46-94cf-b115e698abcb'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.246109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] The token cache is being invalidate because project ab220316e3a149f9ac935be2e50508c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:06.246402 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.247144 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.248921 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1414/2858] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:06 2022] DELETE /identity/v3/projects/ab220316e3a149f9ac935be2e50508c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:06.253689 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:06.265682 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91693e-2f70-41f9-8f34-03f0fcc0f5f1 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:06.333092 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:06.334170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:06.334447 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:06.334741 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] PATH_INFO: `/v3/users/91ffefb7e9ab47a8a2847706ec3ebbe8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:06.336133 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ffefb7e9ab47a8a2847706ec3ebbe8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:06.338083 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:06.399014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ffefb7e9ab47a8a2847706ec3ebbe8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.399789 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ffefb7e9ab47a8a2847706ec3ebbe8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.405414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:06.408084 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ffefb7e9ab47a8a2847706ec3ebbe8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.423414 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ffefb7e9ab47a8a2847706ec3ebbe8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.432810 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-eaf2e65e-2776-4045-9028-7e5a94cb8bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ffefb7e9ab47a8a2847706ec3ebbe8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.439199 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1445/2859] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:06 2022] DELETE /identity/v3/users/91ffefb7e9ab47a8a2847706ec3ebbe8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:06.738242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:06.740037 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:06.753847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db3b8-7577-4c46-94cf-b115e698abcb admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:06.817151 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:06.818083 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:06.818414 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:06.818692 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] PATH_INFO: `/v3/projects/b8e5b4c1093b4a9394d28bd194c51087` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:06.820068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e5b4c1093b4a9394d28bd194c51087)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:06.822551 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:06.874230 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e5b4c1093b4a9394d28bd194c51087', 'request_id': 'req-cd9aebfc-93b2-43b5-8571-889592071346'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.887253 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e5b4c1093b4a9394d28bd194c51087', 'request_id': 'req-cd9aebfc-93b2-43b5-8571-889592071346'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.898378 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e5b4c1093b4a9394d28bd194c51087', 'request_id': 'req-cd9aebfc-93b2-43b5-8571-889592071346'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.912703 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] The token cache is being invalidate because project b8e5b4c1093b4a9394d28bd194c51087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:06.913103 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.914291 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-cd9aebfc-93b2-43b5-8571-889592071346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:06.916194 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1415/2860] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:06 2022] DELETE /identity/v3/projects/b8e5b4c1093b4a9394d28bd194c51087 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:06.921350 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:06.921962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:06.921962 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:06.942186 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:06.943491 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:07.012020 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1446/2861] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:07.076779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.077034 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.077271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.098632 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:07.099997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:07.109573 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1416/2862] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:07.114139 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.125341 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bd00f-2b29-487f-b1e1-667a0a6b8b73 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.131213 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.131887 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.132166 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.132436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.133206 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:59:07.133510 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.135485 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.147773 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1447/2863] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:59:07.152497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.162548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda92c99-9839-448b-8c4c-97e223778a43 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.168771 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.169428 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.169722 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.169997 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.170701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.172662 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.188567 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:07.206404 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1417/2864] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:07.211258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.222536 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d7b8-ca00-42e0-8fd8-7c2d10db14d8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.283172 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.283853 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.284140 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.284419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.285132 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.287267 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.314780 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1448/2865] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:59:07.319119 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.330277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c3c47-155f-480c-9ae7-e3e119a8acc1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.335689 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.336372 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.336657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.336932 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.337675 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.339296 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.350869 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1418/2866] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:07.355285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.366241 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ea044-412e-453a-8f41-d2e58fc6ad72 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.373029 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.373756 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.374080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.374399 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.375246 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.377859 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd372a-6f27-4379-acd1-a881021fcab4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.392483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1449/2867] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:07.396996 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.407548 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477875-4ddd-4a47-8e82-587074be1678 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.413739 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.415040 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.415445 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.415752 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] PATH_INFO: `/v3/projects/e69305e71a03482f924da59a7889e57b/users/349c5e264d0e4bdcb573709cdc7b5cf0/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.432905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69305e71a03482f924da59a7889e57b, user_id=349c5e264d0e4bdcb573709cdc7b5cf0, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.435610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.452314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1419/2868] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:07 2022] PUT /identity/v3/projects/e69305e71a03482f924da59a7889e57b/users/349c5e264d0e4bdcb573709cdc7b5cf0/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:07.462960 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.463302 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.463757 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.493586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] MFA Rules not processed for user `349c5e264d0e4bdcb573709cdc7b5cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:07.494838 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:07.539758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:07.586181 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1450/2869] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:07.590731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.664712 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d3af-4d75-4b84-9487-4a36eb91138f admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.670671 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.671396 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.671680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.671944 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.672651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.674576 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.710766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1420/2870] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:07.714826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.726961 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0905ce-a509-4eec-956c-5feb259a2117 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.789866 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.790548 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.790826 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.791119 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.791817 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.793783 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.818713 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1451/2871] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:07 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:07.824610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.839036 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355df9f-22d2-461d-99cb-192334cbfe68 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.849806 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.851620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.851620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.851620 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.851951 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.853615 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.869766 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1421/2872] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:07.874718 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.885646 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c418d0-96f0-42c6-ae28-3dcb05ccd5e5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:07.890463 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:07.891372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:07.891650 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:07.891919 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] PATH_INFO: `/v3/projects/21db3c2af4e84a6fbaaaedbeef741383/users/22ec10fdc3cc409c80f78b360fc22ba5/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:07.907800 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21db3c2af4e84a6fbaaaedbeef741383, user_id=22ec10fdc3cc409c80f78b360fc22ba5, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:07.910444 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:07.927372 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1452/2873] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:07 2022] PUT /identity/v3/projects/21db3c2af4e84a6fbaaaedbeef741383/users/22ec10fdc3cc409c80f78b360fc22ba5/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:07.932808 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:07.944701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef24b-c349-483e-8851-4157d4af31f1 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:08.013991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:08.014680 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.014960 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.015265 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.016003 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:08.018757 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d2dea-5efb-43b3-bd27-de67f13671f3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:08.033593 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1422/2874] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:08.038260 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:08.049460 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72a72-e89e-420c-a21c-e41c5542a67a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:08.055477 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:08.056385 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.056632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.056935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] PATH_INFO: `/v3/projects/21db3c2af4e84a6fbaaaedbeef741383/users/22ec10fdc3cc409c80f78b360fc22ba5/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.059010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=21db3c2af4e84a6fbaaaedbeef741383, user_id=22ec10fdc3cc409c80f78b360fc22ba5, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:08.061688 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:08.074745 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1453/2875] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:08 2022] PUT /identity/v3/projects/21db3c2af4e84a6fbaaaedbeef741383/users/22ec10fdc3cc409c80f78b360fc22ba5/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:08.085616 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.085919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.086199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.115582 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] MFA Rules not processed for user `22ec10fdc3cc409c80f78b360fc22ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:08.116733 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:08.140022 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:08.203042 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1423/2876] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:08.214875 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:08.216993 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:08.283664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab1ace-8646-4fb6-a068-b0b1dcd96bbe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:08.297102 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:08.297787 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.298075 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.298351 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.309723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:08.311264 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc76438-5191-4403-8161-2d3c680dcd52 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:08.321808 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1454/2877] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:08.470063 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:08.538646 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a1b90-15b3-45f3-97d0-3c5265de913e None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:08.543863 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:08.544824 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.545109 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.545387 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] PATH_INFO: `/v3/users/349c5e264d0e4bdcb573709cdc7b5cf0` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.546657 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349c5e264d0e4bdcb573709cdc7b5cf0)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:08.548713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:08.601780 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349c5e264d0e4bdcb573709cdc7b5cf0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:08.602769 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349c5e264d0e4bdcb573709cdc7b5cf0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:08.612314 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349c5e264d0e4bdcb573709cdc7b5cf0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:08.623356 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349c5e264d0e4bdcb573709cdc7b5cf0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:08.642330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349c5e264d0e4bdcb573709cdc7b5cf0'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:08.648876 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1424/2878] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:08 2022] DELETE /identity/v3/users/349c5e264d0e4bdcb573709cdc7b5cf0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:08.654059 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.654479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.654870 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.676215 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:08.677169 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:08.700926 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:08.748028 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1455/2879] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:08.758992 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:08.760870 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:08.802517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:08.828491 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb0e29-f15b-4f58-b475-286af9c5f5c8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:08.832520 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:08.833176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:08.833460 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:08.833731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:08.844277 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:08.845732 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:08.856068 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1425/2880] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:09.412248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:09.414219 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:09.427184 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f821933-919a-40b5-9d9a-9e9e0a6a2c77 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:09.434127 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:09.435129 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:09.435476 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:09.435818 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] PATH_INFO: `/v3/projects/e69305e71a03482f924da59a7889e57b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:09.438899 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69305e71a03482f924da59a7889e57b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:09.442082 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:09.498015 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69305e71a03482f924da59a7889e57b', 'request_id': 'req-899067da-8b6e-4b1a-8db2-74380b243f2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.506829 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69305e71a03482f924da59a7889e57b', 'request_id': 'req-899067da-8b6e-4b1a-8db2-74380b243f2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.515406 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69305e71a03482f924da59a7889e57b', 'request_id': 'req-899067da-8b6e-4b1a-8db2-74380b243f2d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.530828 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] The token cache is being invalidate because project e69305e71a03482f924da59a7889e57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:09.531124 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.531849 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.533492 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1456/2881] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:09 2022] DELETE /identity/v3/projects/e69305e71a03482f924da59a7889e57b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:09.541711 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:09.552497 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041dd713-9c32-4f9c-93e8-20eedd07a233 service neutron] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:09.614242 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:09.615316 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:09.615608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:09.615891 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] PATH_INFO: `/v3/users/22ec10fdc3cc409c80f78b360fc22ba5` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:09.617117 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ec10fdc3cc409c80f78b360fc22ba5)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:09.619077 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:09.673176 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ec10fdc3cc409c80f78b360fc22ba5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.674353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ec10fdc3cc409c80f78b360fc22ba5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.683716 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ec10fdc3cc409c80f78b360fc22ba5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.692379 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ec10fdc3cc409c80f78b360fc22ba5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.711651 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-02ff0007-5fc3-4ce0-9778-91423204f624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ec10fdc3cc409c80f78b360fc22ba5'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:09.718693 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1426/2882] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:09 2022] DELETE /identity/v3/users/22ec10fdc3cc409c80f78b360fc22ba5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:59:10.012285 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.014143 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:10.037513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899067da-8b6e-4b1a-8db2-74380b243f2d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.103680 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.104568 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.104850 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.105123 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] PATH_INFO: `/v3/projects/21db3c2af4e84a6fbaaaedbeef741383` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.106353 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21db3c2af4e84a6fbaaaedbeef741383)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.108340 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.165258 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:10.167470 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21db3c2af4e84a6fbaaaedbeef741383', 'request_id': 'req-b196ffe6-7e8a-4898-a66d-e681fd109050'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:10.178511 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21db3c2af4e84a6fbaaaedbeef741383', 'request_id': 'req-b196ffe6-7e8a-4898-a66d-e681fd109050'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:10.186154 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21db3c2af4e84a6fbaaaedbeef741383', 'request_id': 'req-b196ffe6-7e8a-4898-a66d-e681fd109050'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:10.197472 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] The token cache is being invalidate because project 21db3c2af4e84a6fbaaaedbeef741383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:10.197759 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:10.198506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b196ffe6-7e8a-4898-a66d-e681fd109050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:10.200216 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1457/2883] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:10 2022] DELETE /identity/v3/projects/21db3c2af4e84a6fbaaaedbeef741383 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:10.205516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.206026 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.206154 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.226682 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:10.228009 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:10.279850 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:10.296902 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1427/2884] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:10.305334 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.305632 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.305914 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.326002 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:10.327428 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:10.338539 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1458/2885] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:10.343143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.353698 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89491010-ebf1-44e1-b40a-dff0f88c7c24 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.358889 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.359598 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.359883 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.360160 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.360903 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80340) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:59:10.361208 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.362839 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.375390 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1428/2886] 10.222.0.74 () {58 vars in 1152 bytes} [Thu Jun 16 16:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:59:10.379880 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.392345 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d3d9e-e2a2-4da7-87b0-61359fcf9cb1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.398049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.398721 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.398999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.399303 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.399999 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.401952 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.432306 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1459/2887] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:10.436916 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.447523 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f79a17-cc4f-4521-95d4-297dddc303b0 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.522838 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.523575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.523851 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.524123 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.524830 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.526753 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.557751 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1429/2888] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:10.562176 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.573827 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9d8a0-6ab1-4274-999f-8c4d1bf8aacd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.580017 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.580888 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.581212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.581529 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.582521 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.584995 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.596360 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1460/2889] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:10.600640 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.611790 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c21a-30cb-4f90-9066-7763e6f11cb7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.619847 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.620779 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.621135 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.621584 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.622644 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.624737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ab6e0-143f-47ac-9ff2-778bf89fbda2 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.637071 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1430/2890] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:10.641714 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.652907 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9427f4-9e96-45d8-9c61-59400f58e5f0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.658814 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.659768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.660051 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.660330 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] PATH_INFO: `/v3/projects/737e52f8900c4cfb95e642cf41af4775/users/d55555e8abcc496c836f2307afb49947/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.676175 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=737e52f8900c4cfb95e642cf41af4775, user_id=d55555e8abcc496c836f2307afb49947, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.678901 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.693723 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1461/2891] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:10 2022] PUT /identity/v3/projects/737e52f8900c4cfb95e642cf41af4775/users/d55555e8abcc496c836f2307afb49947/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:10.710674 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.710963 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.711300 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.740368 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] MFA Rules not processed for user `d55555e8abcc496c836f2307afb49947`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:10.741627 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:10.816308 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1431/2892] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:10.821419 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.895328 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5afcb-3025-46ae-88c3-ae7885b6a826 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:10.902310 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:10.903327 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:10.903690 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:10.904016 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:10.904985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:10.908105 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:10.938489 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1462/2893] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:10.943967 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:10.955403 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807e240-f2a3-4e93-b467-0ef3c3c9544b None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.005978 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:11.017537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.018228 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.018516 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.018791 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.019592 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.021575 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.050020 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1432/2894] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:10 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:11.054117 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.065080 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224f10c-27c0-4de0-8e0b-97e9f6592d2d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.070200 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.070626 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.070935 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.071438 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.072014 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.073586 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.086430 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1463/2895] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:11.090538 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.101106 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cddae-f088-4fe2-9496-f769c9bf5fc3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.106788 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.107531 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.107811 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.108081 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.108804 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.110424 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee85b8-690d-4a62-abee-841086bb5fc1 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.121261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1433/2896] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:11.125558 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.137248 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7118384-0a0e-4632-99c4-bef8bee9376b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.142397 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.143317 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.143600 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.143873 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] PATH_INFO: `/v3/projects/84ff127b1c304f3fa08ebb27dfc75f35/users/0ecde702e4814146935889b5f06e61f9/roles/a0c9d2da81de41eb955b1392d9a2e766` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.160656 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ff127b1c304f3fa08ebb27dfc75f35, user_id=0ecde702e4814146935889b5f06e61f9, role_id=a0c9d2da81de41eb955b1392d9a2e766)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.163366 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.179892 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1464/2897] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:11 2022] PUT /identity/v3/projects/84ff127b1c304f3fa08ebb27dfc75f35/users/0ecde702e4814146935889b5f06e61f9/roles/a0c9d2da81de41eb955b1392d9a2e766 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:11.191145 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.191434 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.191713 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.217994 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] MFA Rules not processed for user `0ecde702e4814146935889b5f06e61f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:11.219249 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:11.292895 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1434/2898] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:11.299753 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.328001 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:11.367715 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df4cb-f3fe-4bed-b56d-c295bd4120f3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.373339 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.374010 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.374295 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.374572 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.375294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.377212 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.405338 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1465/2899] 10.222.0.74 () {60 vars in 1152 bytes} [Thu Jun 16 16:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:11.409695 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.422613 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57c273-8590-464c-be7d-c4e4e1fb423f None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.493758 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.494442 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.494730 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.495008 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] PATH_INFO: `/v3/users` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.495747 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.497706 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.520165 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1435/2900] 10.222.0.74 () {60 vars in 1143 bytes} [Thu Jun 16 16:59:11 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:59:11.524513 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.534916 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1626-91a5-48d4-aaef-25f8cb3296e3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.539833 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.540509 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.540782 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.541060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.541804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.543506 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.558060 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1466/2901] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:11.562342 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.573335 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e947f-7bee-436a-959e-30434ed6e419 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.578489 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.579353 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.579635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.579899 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] PATH_INFO: `/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4/users/c69109614acb45b8add55a3afffaf4d0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.594959 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea8eadf0acb47c3b29ed191e3a98de4, user_id=c69109614acb45b8add55a3afffaf4d0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.597631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.611939 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1436/2902] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:11 2022] PUT /identity/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4/users/c69109614acb45b8add55a3afffaf4d0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:11.615950 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.626823 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c3a6b-fe7a-488f-824e-cef01ec882cd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.684588 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.685259 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.685544 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.685819 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.686554 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.688191 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411acc5a-08ec-48ce-8216-6a36a0b6c6fd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.698369 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1467/2903] 10.222.0.74 () {58 vars in 1121 bytes} [Thu Jun 16 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:59:11.702630 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.714212 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce703f11-ebb7-4976-93c0-11d520388742 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.720435 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.721328 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.721608 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.721881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] PATH_INFO: `/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4/users/c69109614acb45b8add55a3afffaf4d0/roles/279eea356bbd45f39595ccaa1656385a` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.724225 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea8eadf0acb47c3b29ed191e3a98de4, user_id=c69109614acb45b8add55a3afffaf4d0, role_id=279eea356bbd45f39595ccaa1656385a)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:11.727143 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:11.738280 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1437/2904] 10.222.0.74 () {60 vars in 1482 bytes} [Thu Jun 16 16:59:11 2022] PUT /identity/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4/users/c69109614acb45b8add55a3afffaf4d0/roles/279eea356bbd45f39595ccaa1656385a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:11.748479 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.748768 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.749048 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:11.777294 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] MFA Rules not processed for user `c69109614acb45b8add55a3afffaf4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:11.779652 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:11.851966 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1468/2905] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:11.867874 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:11.869854 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:11.885440 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:11.940991 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dcc6e-ccb2-4f7b-af51-de1fa12815d3 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:11.944728 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:11.945537 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:11.945821 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:11.946098 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:12.009332 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:12.011161 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:12.022852 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1438/2906] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:13.578095 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:13.579664 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:13.593777 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07cefa-c4dd-4f07-a148-e10fedca7982 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:13.598236 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:13.598897 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:13.599240 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:13.599500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:13.618453 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:13.678746 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:13.681006 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:13.691275 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1469/2907] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:14.376735 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:14.388070 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e419f18-4baa-432d-b751-ddba6e701dc0 service cinder] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:14.392535 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a66c447d6e943a29b2bf8a6c5309294', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:14.393189 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] REQUEST_METHOD: `GET` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:14.393470 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:14.393750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:14.407141 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:14.408764 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6a952-3045-4748-b445-c2f2158650ce service cinder] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:14.426457 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1439/2908] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:18.365595 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:18.436466 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbbbb5-0841-4ca8-9a10-a9d530e8fec0 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:18.440883 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:18.441769 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:18.442049 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:18.442318 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] PATH_INFO: `/v3/users/d55555e8abcc496c836f2307afb49947` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:18.443567 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55555e8abcc496c836f2307afb49947)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:18.445644 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:18.499034 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55555e8abcc496c836f2307afb49947'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:18.499728 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55555e8abcc496c836f2307afb49947'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:18.507483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55555e8abcc496c836f2307afb49947'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:18.523987 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55555e8abcc496c836f2307afb49947'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:18.540183 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55555e8abcc496c836f2307afb49947'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:18.547425 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1470/2909] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:18 2022] DELETE /identity/v3/users/d55555e8abcc496c836f2307afb49947 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:18.552750 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] REQUEST_METHOD: `POST` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:18.553202 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:18.553631 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:18.572030 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.auth.core [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] MFA Rules not processed for user `55610df78e6e43caada373cf55f4eeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80340) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:59:18.573261 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:18.633913 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1440/2910] 10.222.0.74 () {58 vars in 957 bytes} [Thu Jun 16 16:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:59:18.644890 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:18.647128 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:18.692224 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:18.711724 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622cea0-63d7-430d-8290-bee06f8ebaff admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:18.716249 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db42ab9d8ff4cd2931f40085f983216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:18.716911 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:18.717192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:18.717469 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:18.727913 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:18.729500 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:18.739416 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1471/2911] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 16:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:59:19.026540 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:19.028373 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:19.041587 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daa47a-2c4f-4dee-9f18-12789afad7b5 None None] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:19.048299 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:19.049393 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:19.049701 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:19.049982 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] PATH_INFO: `/v3/projects/737e52f8900c4cfb95e642cf41af4775` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:19.051293 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=737e52f8900c4cfb95e642cf41af4775)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:19.053900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:19.104043 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:19.105952 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737e52f8900c4cfb95e642cf41af4775', 'request_id': 'req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.112901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737e52f8900c4cfb95e642cf41af4775', 'request_id': 'req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.121881 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737e52f8900c4cfb95e642cf41af4775', 'request_id': 'req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.134307 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] The token cache is being invalidate because project 737e52f8900c4cfb95e642cf41af4775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:19.134604 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.135609 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.137199 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1441/2912] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:19 2022] DELETE /identity/v3/projects/737e52f8900c4cfb95e642cf41af4775 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:19.141326 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:19.151654 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e1d86-1c49-49ab-aead-839d3e271708 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:19.212687 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:19.213583 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:19.213874 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:19.214170 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] PATH_INFO: `/v3/users/0ecde702e4814146935889b5f06e61f9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:19.215434 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecde702e4814146935889b5f06e61f9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:19.217370 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:19.268636 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecde702e4814146935889b5f06e61f9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.269352 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecde702e4814146935889b5f06e61f9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.276755 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecde702e4814146935889b5f06e61f9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.288758 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecde702e4814146935889b5f06e61f9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.296716 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecde702e4814146935889b5f06e61f9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.302805 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1472/2913] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:19 2022] DELETE /identity/v3/users/0ecde702e4814146935889b5f06e61f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:19.616600 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:19.618737 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:19.634155 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5eaab-b8e1-4c57-9f8b-6c1525ea4eb7 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:19.700304 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:19.701184 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:19.701464 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:19.701731 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] PATH_INFO: `/v3/projects/84ff127b1c304f3fa08ebb27dfc75f35` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:19.702966 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ff127b1c304f3fa08ebb27dfc75f35)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:19.704900 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:19.760463 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ff127b1c304f3fa08ebb27dfc75f35', 'request_id': 'req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.766517 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ff127b1c304f3fa08ebb27dfc75f35', 'request_id': 'req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.776931 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ff127b1c304f3fa08ebb27dfc75f35', 'request_id': 'req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.788976 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] The token cache is being invalidate because project 84ff127b1c304f3fa08ebb27dfc75f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:19.789271 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.789919 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.791610 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1442/2914] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:19 2022] DELETE /identity/v3/projects/84ff127b1c304f3fa08ebb27dfc75f35 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:19.797207 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:19.799286 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:19.815985 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61d2f4-e40e-4ebc-b463-17fe71e82f00 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:19.879502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:19.886269 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:19.887192 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:19.887483 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:19.887763 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] PATH_INFO: `/v3/users/c69109614acb45b8add55a3afffaf4d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:19.889046 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69109614acb45b8add55a3afffaf4d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:19.890997 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:19.943804 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69109614acb45b8add55a3afffaf4d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.944842 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69109614acb45b8add55a3afffaf4d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.958375 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69109614acb45b8add55a3afffaf4d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.975436 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69109614acb45b8add55a3afffaf4d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.985981 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69109614acb45b8add55a3afffaf4d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:19.994934 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1473/2915] 10.222.0.74 () {58 vars in 1223 bytes} [Thu Jun 16 16:59:19 2022] DELETE /identity/v3/users/c69109614acb45b8add55a3afffaf4d0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:20.311375 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Authenticating user token {{(pid=80340) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:59:20.313901 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.fernet_utils [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80340) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:59:20.334330 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e99c4-52af-4a07-9011-c9b4d23eb8a8 admin admin] Validating token access rules against request {{(pid=80340) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:59:20.407431 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55610df78e6e43caada373cf55f4eeb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0df30bd71bf4670afbc831b314be545', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80340) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:59:20.408417 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:59:20.408635 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:59:20.408905 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] PATH_INFO: `/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4` {{(pid=80340) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:59:20.410294 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea8eadf0acb47c3b29ed191e3a98de4)` {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:59:20.412382 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] RBAC: Authorization granted {{(pid=80340) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:59:20.422183 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:20.471167 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea8eadf0acb47c3b29ed191e3a98de4', 'request_id': 'req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:20.478050 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea8eadf0acb47c3b29ed191e3a98de4', 'request_id': 'req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:20.487348 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea8eadf0acb47c3b29ed191e3a98de4', 'request_id': 'req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429'} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:20.500082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] The token cache is being invalidate because project cea8eadf0acb47c3b29ed191e3a98de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80340) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:59:20.500371 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:20.501082 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG keystone.notifications [None req-e8a3f74e-4989-4bd4-a19d-f1dbd0039429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80340) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:59:20.502969 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: [pid: 80340|app: 0|req: 1443/2916] 10.222.0.74 () {58 vars in 1232 bytes} [Thu Jun 16 16:59:20 2022] DELETE /identity/v3/projects/cea8eadf0acb47c3b29ed191e3a98de4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:59:29.989400 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:59:30.496642 dv-cinder-837355-2-iscsi devstack@keystone.service[80340]: DEBUG dbcounter [-] [80340] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80340) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:03:55.821342 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:03:55.823027 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:03:55.974531 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b5baa-294e-48df-851d-3283a62324e3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:03:56.043720 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b84d7ac4698432e98a30fd39de251f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c245ad526c074bffaaa04e41f9f74e55', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:03:56.044381 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:03:56.044662 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:03:56.044937 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:03:56.046795 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:03:56.059005 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:03:56.060653 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355c9ae-e6dc-4763-8a78-f42b9b210b3f service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:03:56.071502 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1474/2917] 10.222.0.74 () {60 vars in 1365 bytes} [Thu Jun 16 17:03:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:04:06.066927 dv-cinder-837355-2-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}